# LLM Tokens and Context Windows: Count, Budget and Avoid Truncation

> Learn what tokens and context windows are, how to count tokens before a request with the Claude token counting endpoint or tiktoken, and how to stay inside context limits.

Source: https://www.epifive.com/blog/llm-token-counting-context-window-limits · Published 2026-10-25 · EpiFive

## Key takeaways

- A token is the unit a model reads and writes, and the context window is the total number of tokens it can use for input plus output in one request.
- Count Claude tokens before sending with client.messages.count_tokens, which is free and accepts the same system, messages and tools as a normal request.
- The Claude count is an estimate, and models from Claude 4.7 onward tokenize the same text into roughly 30 percent more tokens than earlier models.
- If input alone exceeds the window the Claude API returns a 400 prompt is too long error, so budget input, tools and max_tokens together.
- Use tiktoken only for OpenAI encodings, and read the usage field in every API response to check real consumption.

A token is the chunk of text a language model reads and writes, and the context window is the maximum number of tokens it can handle in one request, counting your input and the model's output together. To stay inside the limit, count tokens before you send a request: call `client.messages.count_tokens` for Claude, use the `tiktoken` library for OpenAI encodings, and check the `usage` data that every response returns.

## Who this is for and what you will have

This guide is for developers who call LLM APIs and want predictable behaviour for long prompts, documents and conversations. By the end you will know what counts toward the window, how to count tokens for Claude and OpenAI-style encodings, how to budget `max_tokens`, and which errors to expect when a request is too large.

## Prerequisites

- Python with the `anthropic` package and an `ANTHROPIC_API_KEY` environment variable for the Claude examples.
- Optionally the `tiktoken` package for OpenAI encodings.

```bash
pip install anthropic tiktoken
```

## What a token is and why the count matters

Models do not read characters or words. They read tokens, which are pieces of text produced by a tokenizer. The `tiktoken` documentation describes its byte pair encoding as reversible and lossless and gives a rough rule that each token corresponds to about 4 bytes on average. Treat that as an order-of-magnitude guide only, because code, non-English text and unusual formatting tokenize differently.

Token counts matter for three reasons: they set what fits in the context window, they drive cost because APIs bill by tokens, and they feed rate limits. Different model families use different tokenizers, so the same text has different counts on different models. For cost calculations see [Claude API Pricing: Estimate Token Costs Before You Build](https://www.epifive.com/blog/claude-api-pricing-token-cost-estimate).

## What counts toward the context window

The Claude documentation defines the context window as all the text a model can reference when generating a response, including the response itself. Everything in the request counts:

- The system prompt.
- Every message in `messages`, including tool results, images and documents.
- Your tool definitions.
- The output Claude generates for the turn, including extended thinking.

Across turns, each user message and assistant response accumulates, and earlier turns are preserved completely unless you manage them. With prompt caching, the input count is split across `input_tokens`, `cache_read_input_tokens` and `cache_creation_input_tokens`, and all three count toward the window. Caching changes what you pay for those tokens, not whether they count.

### Window sizes

According to the Claude documentation, current models including Claude Opus 5.5, Sonnet 5.5, Haiku 5.5 and the Fable 5 family have a 1M-token context window, and a single request can generate up to 128k output tokens (`max_tokens`). Other Claude models, including Claude Sonnet 4.5, have a 200k-token window. Model lists change, so confirm the figure for your model in the model comparison table before you hard-code a limit.

A larger window is not automatically better. The documentation notes that as token count grows, accuracy and recall degrade, which it calls context rot, so curating what goes into the request matters as much as how much space remains.

### Thinking tokens

With extended or adaptive thinking, thinking tokens count toward the context window and are a subset of `max_tokens`. They are billed as output tokens. Whether thinking blocks from earlier assistant turns stay in context depends on the model: some keep them by default and they count as input, and others strip them automatically when you pass them back. Check the thinking documentation for your model.

## Count Claude tokens before you send

The token counting endpoint accepts the same structured inputs as creating a message, including system prompts, tools, images and PDFs, and returns the total number of input tokens.

```python
import anthropic

client = anthropic.Anthropic()

response = client.messages.count_tokens(
    model="claude-opus-5-5",
    system="You are a scientist",
    messages=[{"role": "user", "content": "Hello, Claude"}],
)

print(response.json())
```

The documentation shows this producing `{ "input_tokens": 14 }`. Pass the model you will actually use, because the endpoint counts with that model's tokenizer.

### Count tools

Tool definitions consume tokens on every request, so include them when you count.

```python
response = client.messages.count_tokens(
    model="claude-opus-5-5",
    tools=[
        {
            "name": "get_weather",
            "description": "Get the current weather in a given location",
            "input_schema": {
                "type": "object",
                "properties": {
                    "location": {
                        "type": "string",
                        "description": "The city and state, e.g. San Francisco, CA",
                    }
                },
                "required": ["location"],
            },
        }
    ],
    messages=[{"role": "user", "content": "What's the weather like in San Francisco?"}],
)

print(response.json())
```

The documentation's sample output for this request is `{ "input_tokens": 403 }`, which shows how a single small tool adds to a short prompt.

### What the endpoint does not accept

The endpoint returns an `invalid_request_error` for some inputs the Messages API accepts:

- Server tools such as web search, web fetch, code execution and tool search, with the advisor tool as the exception.
- The MCP connector.
- `image` or `document` blocks with a `url` or `file` source.

Send images and PDFs as base64 to count them. For requests that use server tools or MCP servers, read the `usage` object in the Messages API response instead.

### Limits, cost and accuracy

Token counting is free, but it has requests-per-minute limits by usage tier.

| Usage tier | Requests per minute |
| --- | --- |
| Start | 5,000 |
| Build | 10,000 |
| Scale | 20,000 |

Counting and message creation have separate, independent limits, so one does not consume the other. Two accuracy points matter:

- The count is an estimate and may differ by a small amount from the real usage. It may include system-added tokens that you are not billed for.
- Counting does not use prompt caching. You can include `cache_control` blocks, but caching only happens when you create a message.

### The tokenizer changed

The documentation states that Claude 4.7 and later models, and the Fable and Mythos models, use a newer tokenizer. The same text produces approximately 30 percent more tokens than on earlier models, with the exact increase depending on the content. Do not reuse counts measured on an older model to estimate cost or window fit. To measure the difference for your workload, count the same request against both models and compare the two `input_tokens` values.

## Count OpenAI-style tokens with tiktoken

For OpenAI models, `tiktoken` is the open source tokenizer. Get an encoding by name or by model, then encode the text and take the length.

```python
import tiktoken

enc = tiktoken.get_encoding("o200k_base")
# or: enc = tiktoken.encoding_for_model("gpt-4o")

tokens = enc.encode("hello world")
print(len(tokens))
assert enc.decode(tokens) == "hello world"
```

The `len(enc.encode(text))` pattern is the standard way to count, though the repository README does not show a counting example and points to an OpenAI Cookbook notebook for that. Use `tiktoken` only for OpenAI encodings. It does not count Claude tokens, so use the Claude endpoint for Claude prompts. Because `tiktoken` counts text, confirm real consumption from the `usage` field of the API response rather than relying on text-only counts.

## Budget input, output and tools together

A simple budget for each request is:

```text
available_for_input = context_window - max_tokens
```

Your system prompt, tool definitions, conversation history and retrieved documents must fit in that remainder. A practical routine is:

1. Count the fixed parts once: system prompt and tool definitions.
2. Count the variable parts per request: history, retrieved text, user input.
3. Compare the total plus `max_tokens` with the window and trim or summarise if it does not fit.

```python
CONTEXT_WINDOW = 1_000_000  # confirm for your model
MAX_OUTPUT = 4_096

def fits(client, model, system, messages, tools=None):
    kwargs = {"model": model, "system": system, "messages": messages}
    if tools:
        kwargs["tools"] = tools
    used = client.messages.count_tokens(**kwargs).input_tokens
    return used + MAX_OUTPUT <= CONTEXT_WINDOW, used
```

You rarely need to run this for a 1M window with short chat turns, but it protects document-heavy workflows, and it is essential on models with a 200k window.

### Ways to reduce what you send

- Drop or shorten old turns that no longer matter.
- Retrieve only the relevant passages instead of whole documents.
- Defer rarely used tool definitions. The documentation points to the tool search tool and a guide on managing tool context.
- Use server-side compaction for long conversations. The documentation describes it as automatically summarising earlier parts of the conversation, available in beta for Claude 4.6 and later models.
- Use context editing to clear old tool results or thinking blocks in agentic workflows.

## What happens when you exceed the limit

| Situation | Claude API behaviour |
| --- | --- |
| Input alone exceeds the window | 400 `invalid_request_error` with "prompt is too long", on every model |
| Input plus `max_tokens` exceeds the window, Claude 4.5 and newer | Request accepted. If generation hits the window it stops with `stop_reason: "model_context_window_exceeded"` |
| Same situation on earlier models | A validation error, unless you opt in to the newer behaviour with the `model-context-window-exceeded-2025-08-26` beta header |
| Too many images or PDF pages | Up to 600 per request for 1M-window models and 100 for 200k-window models, and request size limits may be reached before the token limit |

Always check `stop_reason` in your code. A response that stops because of the window or `max_tokens` can be cut off in the middle of a sentence or a JSON document, so treat it as incomplete. If you parse JSON from the output, see [Claude Structured Outputs: Get Valid JSON from the API in Python](https://www.epifive.com/blog/claude-structured-outputs-json-schema-python) for the related failure cases.

## Troubleshooting

- **"prompt is too long" error.** Count tokens first, then trim history or retrieved text, or compact the conversation.
- **Count differs slightly from billed usage.** The count is an estimate. Compare with the `usage` object from real responses.
- **Counts went up after changing models.** The newer tokenizer on Claude 4.7 and later produces more tokens for the same text. Recount.
- **`invalid_request_error` from count_tokens but not from messages.create.** Your request has a server tool, the MCP connector, or a `url` or `file` source for an image or document.
- **Replies cut off.** `max_tokens` or the window was reached. Raise `max_tokens` if the window allows, or shorten the input.

## Checklist

- Count with the model you will call, not a different one.
- Include system prompt, tools, history, documents and `max_tokens` in the budget.
- Treat counts as estimates and read `usage` from real responses.
- Check `stop_reason` before using output.
- Recount after any model change.
- Use `tiktoken` for OpenAI encodings only.

## References

- Token counting, Claude API documentation: https://platform.claude.com/docs/en/build-with-claude/token-counting
- Context windows, Claude API documentation: https://platform.claude.com/docs/en/build-with-claude/context-windows
- tiktoken repository: https://github.com/openai/tiktoken

## Frequently asked questions

### Does counting tokens with the Claude API cost money?

No. The token counting endpoint is free to use, but it has its own requests-per-minute rate limits by usage tier: 5,000 on Start, 10,000 on Build and 20,000 on Scale. Those limits are separate from message creation limits.

### Is the token count exact?

It is an estimate. The actual number of input tokens used when you create a message can differ by a small amount, and the count may include tokens that Anthropic adds for system optimizations, which you are not billed for.

### What happens when my prompt is longer than the context window?

If the input alone exceeds the window, the Claude API returns a 400 invalid_request_error with the message "prompt is too long". On Claude 4.5 and newer models, a request where input plus max_tokens exceeds the window is accepted and generation stops with stop_reason model_context_window_exceeded.

### Can I reuse token counts across different Claude models?

Not safely. Claude 4.7 and later models use a newer tokenizer that produces roughly 30 percent more tokens for the same text, depending on the content. Count your prompts against the exact model you plan to use.