Prompt Token Counter

What is tokenization?

AI language models don't read text character by character. Instead, they split text into tokens: chunks that are usually a word, a punctuation mark, or a common sub-word sequence.¹ A word like "tokenization" might be one token in a large modern vocabulary, but in an older or smaller vocabulary the same word might split into "token" and "ization", producing two tokens instead of one. The exact split depends on the training data and vocabulary size of each model's tokenizer.²

Why tokens are the billing unit

Tokens are the billing unit for every major AI API, which is why the counter matters as much as the model you pick. At the API level, the context window (the maximum amount of text a model can hold in memory at once) is also measured in tokens rather than characters or words, so both cost and capacity are tracked in the same unit. A 1M-token context window sounds enormous, yet a single dense research paper can exceed 100K tokens once references, tables, and formatting are included.

Because providers bill per token, the same prompt can cost dramatically different amounts depending on how aggressively a model's tokenizer splits it, and that difference compounds across every request in a high-volume pipeline. Counting before you send gives you a hard number to plug into cost estimates instead of relying on rough character-based guesses that often undercount by 30 percent or more.

A short sentence makes the count concrete. Pasting The quick brown fox jumps over the lazy dog. into the box produces exactly 10 tokens for GPT-5.6's o200k_base tokenizer, close to but not identical to the 9-word count a naive split would give you. That gap between word count and token count is the whole reason a dedicated counter exists instead of a rough character-division guess.

Why count before the API call?

The only reliable place to count tokens is client-side, before the request. Once the request is sent, the model's usage field in the response tells you how many tokens were consumed, but by that point you've already paid and potentially hit a context limit.³ For pipelines that process thousands of prompts, knowing the token count before sending lets you route shorter prompts to cheaper models, reject oversized inputs before they cause API errors, and estimate batch costs accurately.³

You cannot ask an AI to count its own tokens before the request is sent. The model has no knowledge of how many tokens it will use until it processes the input. This tool counts tokens locally using the same libraries the APIs themselves use (tiktoken for OpenAI) or close equivalents, so the counts you see match what the API bills.

Reading the context bar

The CONTEXT column shows two things: what percentage of the model's context window your prompt occupies, and a color-coded bar. Green means the prompt fits comfortably. At 75%, the bar turns amber, signaling that you're leaving little room for the model's response. It turns red at 95%, indicating you're near the hard limit; the API may reject the request or silently truncate your input.

How context windows shape the bar

Context windows vary enormously across the 37 models shown: Claude Haiku 4.5 tops out at 200K tokens while GPT-5.6 Sol allows about 1.05M.⁴⁵ That gap means a prompt that barely registers on one model can blow past the limit of another, which is exactly why the context bar is per-model rather than a single number. When a prompt is too long for a particular provider, the grid lets you see at a glance which alternative models still have headroom, so you can decide whether to trim or to switch before committing to an API call.

Why prices differ between providers

Model pricing reflects compute cost, model size, and competitive positioning. Frontier models like Claude Opus 5 and GPT-5.6 Sol charge $5/M input tokens because they run on large, expensive hardware.⁶⁷ Smaller or distilled models like Gemini 3.5 Flash-Lite ($0.30/M input, $2.50/M output) trade some capability for dramatically lower cost.⁸ For tasks that don't need top-tier reasoning, such as summarization, classification, and structured extraction, a cheaper model for the bulk of processing can reduce costs by 10x or more.

The INPUT $ column shows only what it costs to send your prompt, but that is only half the bill. The OUT $/M column is the per-million-token rate charged for the model's response, and for most conversational workloads output tokens run 3 to 6 times more expensive than input tokens, so a long reply can easily cost more than the prompt that triggered it. Multiplying the output rate by your expected reply length and adding the input cost gives you a realistic per-request estimate before you spend a cent.⁸

Prices in this tool are pulled automatically from pydantic/genai-prices, an open-source AI pricing database maintained by the Pydantic team and the community, and refresh at least once per day so the numbers stay current as providers adjust their rates. Because the data comes from a public repository rather than a private integration, you can verify any figure against the source. Thank you to everyone contributing to that project.

Prompt structure and token efficiency

Formatting choices affect token count more than most developers expect. Markdown characters, XML tags used for structure, verbose phrasing, and repeated context all add tokens that cost money on every request. System prompts are charged on every API call. A 500-token system prompt across 10,000 requests costs 5 million input tokens for the system prompt alone, at whatever rate applies to that model.⁹ Auditing your system prompt for repetition and trimming verbose instructions is one of the highest-leverage cost optimizations available for high-volume AI applications.

Turning token awareness into a workflow habit

When your context bar turns red, the fastest fix is usually removing repeated context, shortening instructions, or summarizing earlier conversation turns rather than switching to a more expensive model with a larger window. Counting tokens before the API call, rather than discovering the limit through an error response, is the habit that separates stable production deployments from fragile ones. CapyToolkit runs all tokenization locally in your browser so you can audit prompt structure without sending any content to a server.