# LLM API Timeouts, Retries and Rate Limits: A Python Reliability Guide

> Make Claude and OpenAI API calls in Python resilient: set timeouts, retry only the right errors, honor retry-after on 429, and tell spend limits from rate limits.

Source: https://www.epifive.com/blog/llm-api-timeouts-retries-rate-limits-python · Published 2026-10-31 · EpiFive

## Key takeaways

- Both the Anthropic and OpenAI Python SDKs already retry connection errors, 408, 409, 429 and 5xx twice with backoff, so add your own retries only on top of that deliberately.
- Set a timeout well below the 10-minute SDK default, and use streaming for long generations instead of raising the timeout.
- Honor the retry-after header on a 429, and fall back to exponential backoff with jitter when it is missing.
- Do not retry spend-limit or exhausted-credit 429s, because they keep failing until you change billing or limits.
- Log the request ID from every failed call so you can report it to the provider.

Both the Anthropic and OpenAI Python SDKs already retry connection errors, 408, 409, 429 and 5xx responses twice with short exponential backoff, and both default to a 10-minute timeout. To make LLM calls reliable you need to set a sensible timeout, decide which errors deserve more retries, honor the `retry-after` header, avoid retrying errors that cannot succeed, and log request IDs. This guide shows how, with code for both providers.

It is for Python developers whose LLM-backed feature works in testing but fails under load, on flaky networks or when a billing limit is hit. By the end you will have a small retry wrapper, a timeout policy, a way to classify 429 errors and a checklist. The SDK behavior below comes from the providers' documentation, checked on 31 October 2026. For queues and concurrency control, see [handling LLM API rate limits: retry, backoff and queues in Python](https://www.epifive.com/blog/llm-api-rate-limits-retry-backoff-python). This article focuses on timeouts and error classification. For token budgeting, see [LLM token counting and context window limits](https://www.epifive.com/blog/llm-token-counting-context-window-limits).

## Who this is for and what you will have at the end

You call Claude or OpenAI models from Python services, scripts or workers. You will end with explicit timeouts, a classification of errors into retry and do-not-retry, a wrapper that respects `retry-after`, and logging that makes failures traceable.

## Prerequisites

- Python and the `anthropic` and/or `openai` packages installed, with API keys in environment variables.
- Basic familiarity with exceptions and `try`/`except`.

## What the SDKs already do

Anthropic's Python SDK documentation states that certain errors are automatically retried 2 times by default, with a short exponential backoff. Connection errors, 408 Request Timeout, 409 Conflict, 429 Rate Limit and 5xx errors are all retried. Anthropic's errors page adds that the SDK honors the `retry-after` header when present. The OpenAI Python README gives the same list and default, and says requests are retried only when their body can be safely resent. OpenAI's rate-limits guide adds that official SDKs retry some 429 and 503 errors automatically but you should check your version's behavior.

Both let you change this with `max_retries`. Setting it to 0 disables automatic retries.

```python
from anthropic import Anthropic

client = Anthropic(max_retries=0)          # handle retries yourself
careful = Anthropic()                       # default: 2 retries

# Per-request override, documented for the Anthropic SDK
careful.with_options(max_retries=5).messages.create(
    max_tokens=1024,
    messages=[{"role": "user", "content": "Hello, Claude"}],
    model="claude-sonnet-5-5",
)
```

The decision: leave the built-in retries on and add an outer layer only when you need a longer, deliberate retry budget or custom logging. Do not stack unlimited retries on top of SDK retries, because attempts multiply. With the default 2 SDK retries and 3 attempts in your own wrapper, one logical call can make 9 HTTP requests.

## Set timeouts deliberately

Both SDKs default to 10 minutes. That is far too long for an interactive feature. Both accept a float (seconds) or a granular timeout object.

Anthropic's documentation shows:

```python
import httpx2
from anthropic import Anthropic

client = Anthropic(timeout=20.0)  # 20 seconds for every request

# Granular: overall 60s, read 5s, write 10s, connect 2s
client = Anthropic(
    timeout=httpx2.Timeout(60.0, read=5.0, write=10.0, connect=2.0),
)

# Per-request override
client.with_options(timeout=5.0).messages.create(
    max_tokens=1024,
    messages=[{"role": "user", "content": "Hello, Claude"}],
    model="claude-sonnet-5-5",
)
```

The documentation uses the `httpx2` package for the granular object and states that the `http_client` and timeout objects must come from `httpx2`; a client from the separate `httpx` package raises a `TypeError`. Older SDK versions used `httpx`, so match the import to your installed version's documentation. A plain float works with either.

On timeout the SDK raises `APITimeoutError`, and timed-out requests are retried twice by default. A timeout of 20 seconds with 2 retries can therefore take about a minute before your code sees an error. Include that in your user-facing deadlines.

### Pick timeouts by request type

| Request type | Timeout approach |
|---|---|
| Short classification or extraction | Low, such as 10 to 30 seconds |
| Chat reply shown to a user | Use streaming so first tokens arrive quickly |
| Long generation or large `max_tokens` | Streaming, or the Batches API |
| Offline bulk work | Batch API, not synchronous calls |

The numbers in the first row are starting points for you to tune, not provider recommendations.

### Use streaming for long requests

Anthropic warns against setting a large `max_tokens` without streaming. Some networks drop idle connections, which can make a request fail or time out without a response from the API. The Python SDK raises a `ValueError` if a non-streaming request is expected to take longer than roughly 10 minutes, and passing `stream=True` or overriding `timeout` disables that error. If you do not need incremental output, the SDK can consume the stream for you and return the complete message:

```python
with client.messages.stream(
    max_tokens=4096,
    messages=[{"role": "user", "content": "Write a detailed analysis..."}],
    model="claude-sonnet-5-5",
) as stream:
    message = stream.get_final_message()

print(next(b.text for b in message.content if b.type == "text"))
```

Anthropic also notes that with server-sent events an error can occur after a 200 response, so mid-stream errors need handling separately from status-code errors.

## Classify errors: retry or fail fast

Treat errors in three groups.

| Group | Examples | Action |
|---|---|---|
| Transient | Connection errors, timeouts, 408, 500, 503, 504, 529, rate-limit 429 | Retry with backoff |
| Caller bugs | 400, 401, 403, 404, 413, 422 | Fix the request or credentials; do not retry |
| Account limits | Spend-cap 429, credits exhausted | Alert a person; do not retry |

Anthropic documents these codes: 400 `invalid_request_error`, 401 `authentication_error`, 402 `billing_error`, 403 `permission_error`, 404 `not_found_error`, 413 `request_too_large`, 429 `rate_limit_error`, 500 `api_error`, 504 `timeout_error` and 529 `overloaded_error`. For 500 it says to retry with exponential backoff and contact support with the request ID if it persists. A 529 means the API is temporarily overloaded.

### 429 is not always a rate limit

Both providers use 429 for two different situations.

- **Claude API:** a 429 can mean a rate limit, or that your usage tier's monthly spend cap was reached. The spend-cap 429 has no `retry-after` header and keeps failing until access resumes; on the Messages API its `error.details.error_code` is `enforced_spend_limit_reached`. A spend limit you set yourself returns HTTP 400 with `invalid_request_error` instead.
- **OpenAI:** temporary rate limits are retryable, but `credit_balance_exhausted`, `organization_spend_limit_exceeded`, `project_spend_limit_exceeded` and `organization_usage_limit_exceeded` are not fixed by retrying. OpenAI says to check `error.code`, since `error.type` may only show `insufficient_quota`.

Anthropic also warns that a sharp increase in traffic can trigger 429s from acceleration limits, and recommends increasing traffic gradually and keeping usage consistent. OpenAI describes a similar `slow_down` case.

### Honor retry-after

Anthropic documents `retry-after` as the number of seconds to wait; earlier retries will fail, and not every 429 has it. OpenAI says to wait at least that long when it is present and to use exponential backoff with jitter when it is missing. OpenAI also notes that failed requests still count toward your per-minute limit, so resending immediately does not help.

## A retry wrapper for both providers

This wrapper retries transient failures, respects `retry-after`, skips non-retryable cases and caps both attempts and total time. It treats the SDK's own retries as the first layer, so run it with `max_retries=0` clients if you want to be the only layer.

```python
import json
import random
import time

import anthropic
import openai

RETRYABLE_STATUS = {408, 409, 429, 500, 502, 503, 504, 529}
NON_RETRYABLE_CODES = {
    "enforced_spend_limit_reached",      # Claude API spend cap
    "credit_balance_exhausted",          # OpenAI
    "organization_spend_limit_exceeded",
    "project_spend_limit_exceeded",
    "organization_usage_limit_exceeded",
    "insufficient_quota",
}


def _error_code(exc) -> str | None:
    """Best-effort extraction of the provider's error code from the response body."""
    try:
        body = exc.response.json()
    except (AttributeError, ValueError, json.JSONDecodeError):
        return None
    err = body.get("error", {}) if isinstance(body, dict) else {}
    details = err.get("details") or {}
    return details.get("error_code") or err.get("code") or err.get("type")


def call_with_retries(fn, *, max_attempts=4, max_total_seconds=60.0,
                      base_delay=1.0, max_delay=20.0, log=print):
    """Call fn() and retry transient LLM API failures within a budget."""
    start = time.monotonic()
    for attempt in range(1, max_attempts + 1):
        try:
            return fn()
        except (anthropic.APIConnectionError, anthropic.APITimeoutError,
                openai.APIConnectionError, openai.APITimeoutError) as exc:
            last_exc = exc
            retry_after = None
            reason = type(exc).__name__
        except (anthropic.APIStatusError, openai.APIStatusError) as exc:
            code = _error_code(exc)
            request_id = exc.response.headers.get("request-id") or \
                exc.response.headers.get("x-request-id")
            if exc.status_code not in RETRYABLE_STATUS or code in NON_RETRYABLE_CODES:
                log(f"not retrying status={exc.status_code} code={code} "
                    f"request_id={request_id}")
                raise
            header = exc.response.headers.get("retry-after")
            retry_after = float(header) if header and header.replace(".", "", 1).isdigit() else None
            last_exc = exc
            reason = f"status={exc.status_code} request_id={request_id}"

        if attempt == max_attempts:
            raise last_exc
        delay = retry_after if retry_after is not None else min(
            max_delay, base_delay * 2 ** (attempt - 1))
        delay += random.uniform(0, 0.25 * delay)        # jitter
        if time.monotonic() - start + delay > max_total_seconds:
            raise TimeoutError(
                f"retry budget exhausted after {attempt} attempts ({reason})"
            ) from last_exc
        log(f"attempt {attempt} failed ({reason}); sleeping {delay:.1f}s")
        time.sleep(delay)
```

Usage:

```python
client = anthropic.Anthropic(max_retries=0, timeout=30.0)

reply = call_with_retries(lambda: client.messages.create(
    model="claude-sonnet-5-5",
    max_tokens=512,
    messages=[{"role": "user", "content": "Summarize this ticket in one sentence."}],
))
```

Notes on the code:

- `retry-after` is read as a number of seconds. If a provider sends a date form instead, the code falls back to backoff.
- Timeouts and connection failures are caught explicitly. The documentation names `APITimeoutError` for timeouts and `APIConnectionError` for connection failures, so test the wrapper against your SDK versions before relying on it.
- Request IDs come from the `request-id` header for Anthropic and `x-request-id` for OpenAI, which is what the SDK documentation describes.
- The wrapper does not make non-idempotent side effects safe. If the model call triggers a payment or an email, add your own idempotency key before retrying.

## Reduce the chance of hitting limits

- **Cache repeated input.** For most Claude models only uncached input tokens count toward input-tokens-per-minute limits, so prompt caching raises effective throughput.
- **Set `max_tokens` realistically on OpenAI.** OpenAI says usage counts against TPM based on your `max_tokens` setting as well as input, so set it close to the expected response size. Anthropic states that `max_tokens` does not factor into its output-token rate limit calculations.
- **Use batch APIs for offline work.** OpenAI's Batch API runs large sets of requests without using your synchronous rate limits, and Anthropic offers the Message Batches API with its own limits.
- **Watch the headers.** Claude responses carry `anthropic-ratelimit-requests-remaining`, `anthropic-ratelimit-input-tokens-remaining` and related reset headers. OpenAI returns `x-ratelimit-*` fields. Log them to see headroom before a 429.

## Verify your handling

1. Set a very low timeout such as 0.001 seconds on a test client and confirm you get an `APITimeoutError` and your wrapper logs the retries.
2. Point a client at a local stub that returns 429 with `retry-after: 2` and confirm the wrapper sleeps about two seconds.
3. Return a 401 from the stub and confirm no retry happens.
4. Return a 429 whose body has `error.code` set to a non-retryable value and confirm it fails immediately.

## Troubleshooting

### Requests hang for minutes

You are on the 10-minute default timeout. Set `timeout` and use streaming for long outputs.

### The SDK raises ValueError about expected duration

Anthropic's Python SDK raises it for non-streaming requests expected to run longer than about 10 minutes. Use streaming, the Batches API, or override `timeout` deliberately.

### Retries never succeed on a 429

Check for the spend-cap case. If there is no `retry-after` and the error code is `enforced_spend_limit_reached`, or OpenAI returns a credit or spend-limit code, raise your limit or add credits instead.

### Calls take far longer than your timeout

SDK retries multiply the delay. A 20-second timeout with 2 retries can reach about a minute. Lower `max_retries` or account for it in your deadline.

### 413 request too large

Anthropic lists maximum request sizes of 128 MB for the Messages API. Reduce the payload.

## Checklist

- Set an explicit timeout on every client.
- Use streaming for long generations and Batches for offline work.
- Decide whether the SDK's 2 retries are enough before adding your own layer.
- Retry only transient errors, and honor `retry-after`.
- Never retry spend-cap or exhausted-credit 429s.
- Cap attempts and total retry time, and add jitter.
- Log the request ID of every failure.

## References

- [Claude API errors](https://platform.claude.com/docs/en/api/errors)
- [Claude API rate limits](https://platform.claude.com/docs/en/api/rate-limits)
- [Anthropic Python SDK](https://platform.claude.com/docs/en/cli-sdks-libraries/sdks/python)
- [OpenAI rate limits guide](https://developers.openai.com/api/docs/guides/rate-limits)
- [OpenAI error codes](https://developers.openai.com/api/docs/guides/error-codes)
- [OpenAI Python library](https://github.com/openai/openai-python)

## Frequently asked questions

### Should I retry a 429 error from an LLM API?

Retry a rate-limit 429, waiting at least the retry-after value when the response has it. Do not retry a 429 that means a spend cap or exhausted credits. The Claude API's spend-cap 429 has no retry-after header, and OpenAI uses codes such as credit_balance_exhausted that retrying cannot fix.

### What is the default timeout in the Anthropic and OpenAI Python SDKs?

Both SDK documentation pages say requests time out after 10 minutes by default. You can pass a float number of seconds or a granular timeout object. On timeout the SDK raises APITimeoutError, and timed-out requests are retried twice by default.

### How many times do the SDKs retry by default?

Both retry certain errors 2 times by default with a short exponential backoff: connection errors, 408, 409, 429 and 5xx responses. You can change this with the max_retries option, and setting it to 0 disables automatic retries so you can handle them yourself.

### Why do long LLM requests fail even when the API is healthy?

Some networks drop idle connections after a period of time, which can make a long non-streaming request fail or time out without a response. Anthropic recommends streaming or the Message Batches API for long-running requests, especially those over 10 minutes.