LLM token counter and estimator.

Large language models do not read characters or words. A tokenizer first splits text into tokens: pieces from a fixed vocabulary that the model learned to work with. Most current tokenizers use byte-pair encoding (BPE) or a close relative, with vocabularies of tens of thousands to a few hundred thousand entries.

Common English words are usually one token, often including the space before them. Rare words, names and typos are split into several pieces. Numbers are cut into short chunks, punctuation and indentation take tokens of their own, and text in scripts that were rarer in the tokenizer's training data is broken into many small pieces.

Tokens matter because everything is measured in them: the context window a model can see, the maximum length of its reply, rate limits and the price of each request.

Each model family has its own tokenizer, so only that tokenizer gives an exact count. The estimator uses two widely quoted rules of thumb for English instead: about 4 characters per token, and about 0.75 words per token. It applies both, shows the range between them, and uses the midpoint as the headline figure.

Alongside the estimate you get character, word and line counts. If more than about a fifth of the text is in a non-Latin script, the tool warns that the real count is likely to be higher than the estimate. Everything is simple arithmetic in your browser, so it is instant even for long documents, and nothing is sent to any model or server.

Use the estimate for planning questions: will this document fit, how much room is left for the conversation, roughly what will a batch of requests cost. Do not use it to enforce hard limits in production code, where an error of ten percent can mean a rejected request; count with the real tokenizer there instead.

The same paragraph can produce noticeably different token counts in different model families, because their vocabularies differ. A newer tokenizer with a larger vocabulary often needs fewer tokens for the same text, especially for code and non-English languages.

Content type matters as much as model choice. Source code, JSON, tables, URLs and long numbers need more tokens per character than prose. Hindi, Arabic, Japanese, Thai and many other languages typically need more tokens than English for the same meaning, which means less room in the window and a higher bill.

Requests also carry hidden tokens: chat formatting adds a few per message, tool and function definitions are sent with every call, and images, audio and files are converted into tokens of their own. This estimator counts only the text you paste.

For an exact figure, use the tokenizer that belongs to your model. OpenAI publishes the open-source tiktoken library, which can look up the right encoding from a model name. Anthropic and Google offer token-counting endpoints in their APIs for their models, and open-weight models ship their tokenizer with the weights, usually loadable through Hugging Face's tokenizers or transformers libraries.

The simplest source of truth is the response itself: most LLM APIs return a usage object with the input and output token counts for each request. Log it, and you can compare estimates with reality over time.

The context window holds everything a model sees in one request: the system prompt, tool definitions, retrieved documents, the conversation so far and the reply it writes. If your input already fills most of it, the model has little room to answer, and many chat applications quietly drop older messages to make space.

Models also have a separate limit on output tokens, often much smaller than the window, and reasoning models may spend part of the budget thinking before they answer. Reserve room for the reply before you decide how much context to send.

The presets on this page (8K, 32K, 128K, 200K and 1M tokens) are common window sizes, not claims about any particular model. Check your provider's documentation for the exact limit, or type a custom size.

For English prose, these rough conversions are handy for planning. 100 tokens is about 75 words. 1,000 words is about 1,300 to 1,400 tokens. A dense page of around 500 words is about 650 to 700 tokens. A 128K-token window therefore holds very roughly 96,000 words of plain English, before you leave any room for instructions or the reply.

Treat these as planning figures for English only. Code, tables, JSON and most non-English text use more tokens for the same length, and the only reliable count comes from the model's own tokenizer. When the difference matters, for example near a limit or when estimating cost at scale, measure a representative sample with the real tokenizer and use the ratio you observe.

Trim the system prompt to the instructions that change behaviour. Summarise or drop old conversation turns instead of resending the whole history. Ask for concise, structured output when you do not need prose, since output tokens are usually priced higher than input tokens.

When documents do not fit, retrieval-augmented generation sends only the passages relevant to the question instead of everything. Several providers also offer prompt caching, which makes a repeated prefix such as a long system prompt cheaper and faster on later calls. Our comparison of RAG vs fine-tuning covers when each approach fits.