Prompt Token Counter

What is tokenization?

AI language models don't read text character by character. Instead, they split text into tokens: chunks that are usually a word, a punctuation mark, or a common sub-word sequence.¹ A word like "tokenization" might be one token in a large modern vocabulary, but in an older or smaller vocabulary the same word might split into "token" and "ization", producing two tokens instead of one. The exact split depends on the training data and vocabulary size of each model's tokenizer.²

The tokenizer itself

Behind that process sits a component with a name of its own. The tokenizer is the piece of software that turns your request text into the token sequence the model actually reads, and it ships with a fixed vocabulary built during training. Because the vocabulary is fixed, one model splits the same text the same way every time, while a different model's tokenizer often splits it differently, which is why a token count is always a per-model fact rather than a property of the text itself. That is the reason this grid lays one prompt against 37 models instead of producing a single authoritative number, and the reason a count quoted without its model is nearly meaningless.

This component distinction explains the ~ markers on several rows. Because an exact count requires the provider's tokenizer as runnable code, and not every provider publishes one that runs inside a browser, the grid divides your pasted characters by 3.8 for those models and marks the result approximate. The division lands close for typical English prose, which makes it safe for a rough size check and unsafe for a budget decision. When a number feeds into cost planning or a context-window fit, trust the exact rows first, and read the marked rows as a floor rather than a promise.

Why tokens are the billing unit

Tokens are the billing unit for every major AI API, which is why the counter matters as much as the model you pick. At the API level, the context window (the maximum amount of text a model can hold in memory at once) is also measured in tokens rather than characters or words, so both cost and capacity are tracked in the same unit. A 1M-token context window sounds enormous, yet a single dense research paper can exceed 100K tokens once references, tables, and formatting are included.

Because providers bill per token, the same prompt can cost dramatically different amounts depending on how aggressively a model's tokenizer splits it, and that difference compounds across every request in a high-volume pipeline. Counting before you send gives you a hard number to plug into cost estimates instead of relying on rough character-based guesses that often undercount by 30 percent or more.

A short sentence makes the count concrete. Pasting The quick brown fox jumps over the lazy dog. into the box produces exactly 10 tokens for GPT-5.6's o200k_base tokenizer, close to but not identical to the 9-word count a naive split would give you. That gap between word count and token count is the whole reason a dedicated counter exists instead of a rough character-division guess.

For a usable rule of thumb, plain English prose runs roughly four characters per token, the same calibration family behind this tool's approximate rows. OpenAI's own token guidance gives the estimate directly: one token is approximately four characters, and 100 tokens approximately 75 words, so a 1,000-character passage lands near 260 tokens and a 500-word document near 650 to 700.³ Treat those figures as planning numbers, not guarantees. Sentence length and vocabulary shift the ratio from one text to the next, which is exactly why the paste box exists: instead of multiplying character counts by a guess, you get the exact split for every model in the grid at once.

The rule holds only for prose. Because code and markup split on symbol density rather than word shape, dense source carries more tokens per character than an English paragraph of the same length, and formatting characters are not free either: Markdown fences, XML tags, and JSON braces are all tokens in their own right. A rough character count can therefore understate a code-heavy prompt badly. Use the estimate for sizing drafts and rough budgets, and switch to the exact rows whenever the number has to be right, because the paste box prices the real text rather than the idealized one.

Why count before the API call?

The only reliable place to count tokens is client-side, before the request. Once the request is sent, the model's usage field in the response tells you how many tokens were consumed, but by that point you've already paid and potentially hit a context limit.⁴ For pipelines that process thousands of prompts, knowing the token count before sending lets you route shorter prompts to cheaper models, reject oversized inputs before they cause API errors, and estimate batch costs accurately.⁴

You cannot ask an AI to count its own tokens before the request is sent. The model has no knowledge of how many tokens it will use until it processes the input. This tool counts tokens locally using the same libraries the APIs themselves use (tiktoken for OpenAI) or close equivalents, so the counts you see match what the API bills.

Not every input matches the calibration. The ~ rows divide characters by 3.8, a ratio tuned to typical English prose, and two kinds of paste knock it off. In code, symbols and identifiers split into small pieces, so the same character count carries more tokens than an English paragraph would. With other languages, the relationship between characters, words, and tokens shifts as well³, so text written without Latin letters or without spaces can drift well past the estimate. For either input, read the exact-tokenizer rows as the trustworthy numbers and treat every ~ value as a floor.

This counter reads pasted text and nothing else. It accepts no files and no images, and a multimodal request's image portion is billed by the provider as its own token quantity, derived from the image's size and the detail level you choose rather than from anything a text box could hold.⁵ For an image-bearing prompt, the practical path splits in two: count the text portion here, then read the image input pricing from the provider's own documentation for the specific model, where the per-image cost is laid out directly. No text-paste count can stand in for that number.

Reading the context bar

The CONTEXT column shows two things: what percentage of the model's context window your prompt occupies, and a color-coded bar. Green means the prompt fits comfortably. At 75%, the bar turns amber, signaling that you're leaving little room for the model's response. It turns red at 95%, indicating you're near the hard limit; the API may reject the request or silently truncate your input.

How context windows shape the bar

Context windows vary enormously across the 37 models shown: Claude Haiku 4.5 tops out at 200K tokens while GPT-5.6 Sol allows about 1.05M.⁶⁷ That gap means a prompt that barely registers on one model can blow past the limit of another, which is exactly why the context bar is per-model rather than a single number. When a prompt is too long for a particular provider, the grid lets you see at a glance which alternative models still have headroom, so you can decide whether to trim or to switch before committing to an API call.

What happens at the limit

Two different failures hide behind a full context bar. Through an API, the provider rejects an oversized request with an error before the model processes anything, so the model generates nothing and you must trim or split the input before resending.⁸ Inside chat products, the same overflow usually never surfaces at all: the interface quietly drops the oldest turns so the newest still fit, which trades an error message for a conversation whose early history has simply drifted out of view. The bar exists so you learn which shape you are heading toward before sending, and the red band begins at 95 percent rather than 100 for a reason: the model's reply needs window space of its own.

The window itself is not the only ceiling. Alongside it, providers publish a maximum output-tokens figure for each model, a separate limit on how long any single reply can run, and this grid's CONTEXT column never reports it because the column scopes to the prompt you paste against the input window only.⁶ Reply-length planning therefore belongs in your own estimate rather than in the bar. A prompt resting comfortably in the green can still collide with the output cap when you ask for a long generation, so before committing a request that needs a long reply, check the model's output limit in the provider's documentation alongside its window.

Why prices differ between providers

Model pricing reflects compute cost, model size, and competitive positioning. Frontier models like Claude Opus 5 and GPT-5.6 Sol charge $4 to $5 per million input tokens because they run on large, expensive hardware.⁹¹⁰ Smaller or distilled models like Gemini 3.5 Flash-Lite ($0.30/M input, $2.50/M output) trade some capability for dramatically lower cost.¹¹ For tasks that don't need top-tier reasoning, such as summarization, classification, and structured extraction, a cheaper model for the bulk of processing can reduce costs by 10x or more.

The INPUT $ column shows only what it costs to send your prompt, but that is only half the bill. The OUT $/M column is the per-million-token rate charged for the model's response, and for most conversational workloads output tokens run 3 to 6 times more expensive than input tokens, so a long reply can easily cost more than the prompt that triggered it. Multiplying the output rate by your expected reply length and adding the input cost gives you a realistic per-request estimate before you spend a cent.¹¹

Input tokens versus output tokens

Input tokens and output tokens name the two halves of every request. On the input side sits everything the request carries: your system prompt, the whole visible conversation, any tool results, and the new user turn. Output is what the model generates in reply. Because a chat request submits its full thread as input on every turn, the input side of a long conversation grows with each message you add, and every new turn re-bills the turns already behind it.⁸ That re-entry is why token awareness pays as a habit rather than a one-time check, and why a counter belongs on your desk before the request rather than in your logs after it.

The split also prices your conversation history. Because output rates sit at 3 to 6 times their matching input rates, a long reply can outcost its own prompt, and the accumulated thread bills as input again on every turn that follows. The grid gives you half that arithmetic exactly: paste the prompt and INPUT $ prices it, while OUT $/M waits for your expected reply length. Your estimate of how much the model will say back is the only missing number, and for multi-turn work, the full conversation is the honest thing to paste.

Prices in this tool are pulled automatically from pydantic/genai-prices, an open-source AI pricing database maintained by the Pydantic team and the community, and refresh at least once per day so the numbers stay current as providers adjust their rates. Because the data comes from a public repository rather than a private integration, you can verify any figure against the source. Thank you to everyone contributing to that project.

Prompt structure and token efficiency

Formatting choices affect token count more than most developers expect. Markdown characters, XML tags used for structure, verbose phrasing, and repeated context all add tokens that cost money on every request. System prompts are charged on every API call. A 500-token system prompt across 10,000 requests costs 5 million input tokens for the system prompt alone, at whatever rate applies to that model.¹² Auditing your system prompt for repetition and trimming verbose instructions is one of the highest-leverage cost optimizations available for high-volume AI applications.

Turning token awareness into a workflow habit

When your context bar turns red, the fastest fix is usually removing repeated context, shortening instructions, or summarizing earlier conversation turns rather than switching to a more expensive model with a larger window. Counting tokens before the API call, rather than discovering the limit through an error response, is the habit that separates stable production deployments from fragile ones. CapyToolkit runs all tokenization locally in your browser so you can audit prompt structure without sending any content to a server.