How Many Tokens Is My Prompt? GPT Token Guide

You are about to send a long prompt to an LLM API and two questions decide whether it works: will it fit in the context window, and what will it cost? Both come down to tokens — and guessing them from character counts is where people get burned.

What a token actually is

Models do not read letters; they read tokens — chunks of text produced by a byte-pair encoding (BPE) learned from training data. Common English words are a single token. Rarer words split: “tokenization” is three (token, ization, and friends vary by tokenizer). Numbers, code and non-English text fragment more.

The practical rule of thumb:

• 1 token ≈ 4 characters of English text
• 1 token ≈ ¾ of a word
• 1000 tokens ≈ 750 words ≈ about 3 pages double-spaced

But the rule of thumb is only good for a ballpark. The same paragraph can differ by 20% or more depending on the tokenizer — which is why “how many tokens is my prompt” deserves an exact answer.

Count tokens exactly (for OpenAI models)

OpenAI publishes its tokenizers, so GPT models can be counted to the token. The Token Counter tool runs the exact same BPE ranks the API uses — o200k_base for GPT-5 / GPT-4o, cl100k_base for GPT-4 Turbo / GPT-3.5 — locally in your browser via a WebAssembly-grade tokenizer:

1. Paste your prompt, document or code.
2. Pick the tokenizer your target model uses.
3. Read exact tokens, characters, words and sentences — live as you type.

The rank files (~2 MB) download once and are cached, so after the first use the counting is instant and works offline. Nothing you paste is sent anywhere.

What about Claude and Gemini?

Anthropic and Google do not publish their tokenizers, so an exact public count is not possible for those models. The honest approach is an estimate: Claude and Gemini tokenizers typically land within ±10–20% of OpenAI’s o200k_base on the same English text. The counter offers those modes explicitly labeled as estimates — better to know a range than to trust a character count.

Why this changes your API bill

API pricing is per token, and output tokens cost 3–8× more than input tokens. A prompt that is 2,000 tokens with a 500-token answer is billed very differently from the reverse. Before committing to an integration, model the numbers with the LLM API Cost Calculator: pick a model, enter input/output tokens per request (exact counts from the token counter), set your daily volume, and get per-call, per-month and per-year costs. Every price field is editable, so you can plug in current list prices or your negotiated rates.

Context windows: the other ceiling

Tokens also decide whether your prompt fits. A 128k-token context window is not 128k characters — it is roughly 96,000 words, but code, JSON and non-English text eat it much faster. If your retrieval pipeline stuffs documents into the prompt, count each chunk before designing the budget; the difference between an estimate and an exact count compounds across dozens of chunks.

Quick checklist

1. Count your prompt exactly with the token counter — pick the right tokenizer.
2. Estimate the response length (or measure it from one real call).
3. Price the request and the monthly volume with the cost calculator.
4. Only then decide whether to trim the prompt, upgrade the model, or ship.