LLM API Timeouts, Retries and Rate Limits: A Python Reliability Guide

Both the Anthropic and OpenAI Python SDKs already retry connection errors, 408, 409, 429 and 5xx responses twice with short exponential backoff, and both default to a 10-minute timeout. To make LLM calls reliable you need to set a sensible timeout, decide which errors deserve more retries, honor the retry-after header, avoid retrying errors that cannot succeed, and log request IDs. This guide shows how, with code for both providers.

It is for Python developers whose LLM-backed feature works in testing but fails under load, on flaky networks or when a billing limit is hit. By the end you will have a small retry wrapper, a timeout policy, a way to classify 429 errors and a checklist. The SDK behavior below comes from the providers' documentation, checked on 31 October 2026. For queues and concurrency control, see handling LLM API rate limits: retry, backoff and queues in Python. This article focuses on timeouts and error classification. For token budgeting, see LLM token counting and context window limits.

Who this is for and what you will have at the end

You call Claude or OpenAI models from Python services, scripts or workers. You will end with explicit timeouts, a classification of errors into retry and do-not-retry, a wrapper that respects retry-after, and logging that makes failures traceable.

Prerequisites

• Python and the anthropic and/or openai packages installed, with API keys in environment variables.
• Basic familiarity with exceptions and try/except.

What the SDKs already do

Anthropic's Python SDK documentation states that certain errors are automatically retried 2 times by default, with a short exponential backoff. Connection errors, 408 Request Timeout, 409 Conflict, 429 Rate Limit and 5xx errors are all retried. Anthropic's errors page adds that the SDK honors the retry-after header when present. The OpenAI Python README gives the same list and default, and says requests are retried only when their body can be safely resent. OpenAI's rate-limits guide adds that official SDKs retry some 429 and 503 errors automatically but you should check your version's behavior.

Both let you change this with max_retries. Setting it to 0 disables automatic retries.

from anthropic import Anthropic
 
client = Anthropic(max_retries=0)          # handle retries yourself
careful = Anthropic()                       # default: 2 retries
 
# Per-request override, documented for the Anthropic SDK
careful.with_options(max_retries=5).messages.create(
    max_tokens=1024,
    messages=[{"role": "user", "content": "Hello, Claude"}],
    model="claude-sonnet-5-5",
)

The decision: leave the built-in retries on and add an outer layer only when you need a longer, deliberate retry budget or custom logging. Do not stack unlimited retries on top of SDK retries, because attempts multiply. With the default 2 SDK retries and 3 attempts in your own wrapper, one logical call can make 9 HTTP requests.

Set timeouts deliberately

Both SDKs default to 10 minutes. That is far too long for an interactive feature. Both accept a float (seconds) or a granular timeout object.

Anthropic's documentation shows:

import httpx2
from anthropic import Anthropic
 
client = Anthropic(timeout=20.0)  # 20 seconds for every request
 
# Granular: overall 60s, read 5s, write 10s, connect 2s
client = Anthropic(
    timeout=httpx2.Timeout(60.0, read=5.0, write=10.0, connect=2.0),
)
 
# Per-request override
client.with_options(timeout=5.0).messages.create(
    max_tokens=1024,
    messages=[{"role": "user", "content": "Hello, Claude"}],
    model="claude-sonnet-5-5",
)

The documentation uses the httpx2 package for the granular object and states that the http_client and timeout objects must come from httpx2; a client from the separate httpx package raises a TypeError. Older SDK versions used httpx, so match the import to your installed version's documentation. A plain float works with either.

On timeout the SDK raises APITimeoutError, and timed-out requests are retried twice by default. A timeout of 20 seconds with 2 retries can therefore take about a minute before your code sees an error. Include that in your user-facing deadlines.

Pick timeouts by request type

The numbers in the first row are starting points for you to tune, not provider recommendations.

Use streaming for long requests

Anthropic warns against setting a large max_tokens without streaming. Some networks drop idle connections, which can make a request fail or time out without a response from the API. The Python SDK raises a ValueError if a non-streaming request is expected to take longer than roughly 10 minutes, and passing stream=True or overriding timeout disables that error. If you do not need incremental output, the SDK can consume the stream for you and return the complete message:

with client.messages.stream(
    max_tokens=4096,
    messages=[{"role": "user", "content": "Write a detailed analysis..."}],
    model="claude-sonnet-5-5",
) as stream:
    message = stream.get_final_message()
 
print(next(b.text for b in message.content if b.type == "text"))

Anthropic also notes that with server-sent events an error can occur after a 200 response, so mid-stream errors need handling separately from status-code errors.

Classify errors: retry or fail fast

Treat errors in three groups.

Anthropic documents these codes: 400 invalid_request_error, 401 authentication_error, 402 billing_error, 403 permission_error, 404 not_found_error, 413 request_too_large, 429 rate_limit_error, 500 api_error, 504 timeout_error and 529 overloaded_error. For 500 it says to retry with exponential backoff and contact support with the request ID if it persists. A 529 means the API is temporarily overloaded.

429 is not always a rate limit

Both providers use 429 for two different situations.

• Claude API: a 429 can mean a rate limit, or that your usage tier's monthly spend cap was reached. The spend-cap 429 has no retry-after header and keeps failing until access resumes; on the Messages API its error.details.error_code is enforced_spend_limit_reached. A spend limit you set yourself returns HTTP 400 with invalid_request_error instead.
• OpenAI: temporary rate limits are retryable, but credit_balance_exhausted, organization_spend_limit_exceeded, project_spend_limit_exceeded and organization_usage_limit_exceeded are not fixed by retrying. OpenAI says to check error.code, since error.type may only show insufficient_quota.

Anthropic also warns that a sharp increase in traffic can trigger 429s from acceleration limits, and recommends increasing traffic gradually and keeping usage consistent. OpenAI describes a similar slow_down case.

Honor retry-after

Anthropic documents retry-after as the number of seconds to wait; earlier retries will fail, and not every 429 has it. OpenAI says to wait at least that long when it is present and to use exponential backoff with jitter when it is missing. OpenAI also notes that failed requests still count toward your per-minute limit, so resending immediately does not help.

A retry wrapper for both providers

This wrapper retries transient failures, respects retry-after, skips non-retryable cases and caps both attempts and total time. It treats the SDK's own retries as the first layer, so run it with max_retries=0 clients if you want to be the only layer.

import json
import random
import time
 
import anthropic
import openai
 
RETRYABLE_STATUS = {408, 409, 429, 500, 502, 503, 504, 529}
NON_RETRYABLE_CODES = {
    "enforced_spend_limit_reached",      # Claude API spend cap
    "credit_balance_exhausted",          # OpenAI
    "organization_spend_limit_exceeded",
    "project_spend_limit_exceeded",
    "organization_usage_limit_exceeded",
    "insufficient_quota",
}
 
 
def _error_code(exc) -> str | None:
    """Best-effort extraction of the provider's error code from the response body."""
    try:
        body = exc.response.json()
    except (AttributeError, ValueError, json.JSONDecodeError):
        return None
    err = body.get("error", {}) if isinstance(body, dict) else {}
    details = err.get("details") or {}
    return details.get("error_code") or err.get("code") or err.get("type")
 
 
def call_with_retries(fn, *, max_attempts=4, max_total_seconds=60.0,
                      base_delay=1.0, max_delay=20.0, log=print):
    """Call fn() and retry transient LLM API failures within a budget."""
    start = time.monotonic()
    for attempt in range(1, max_attempts + 1):
        try:
            return fn()
        except (anthropic.APIConnectionError, anthropic.APITimeoutError,
                openai.APIConnectionError, openai.APITimeoutError) as exc:
            last_exc = exc
            retry_after = None
            reason = type(exc).__name__
        except (anthropic.APIStatusError, openai.APIStatusError) as exc:
            code = _error_code(exc)
            request_id = exc.response.headers.get("request-id") or \
                exc.response.headers.get("x-request-id")
            if exc.status_code not in RETRYABLE_STATUS or code in NON_RETRYABLE_CODES:
                log(f"not retrying status={exc.status_code} code={code} "
                    f"request_id={request_id}")
                raise
            header = exc.response.headers.get("retry-after")
            retry_after = float(header) if header and header.replace(".", "", 1).isdigit() else None
            last_exc = exc
            reason = f"status={exc.status_code} request_id={request_id}"
 
        if attempt == max_attempts:
            raise last_exc
        delay = retry_after if retry_after is not None else min(
            max_delay, base_delay * 2 ** (attempt - 1))
        delay += random.uniform(0, 0.25 * delay)        # jitter
        if time.monotonic() - start + delay > max_total_seconds:
            raise TimeoutError(
                f"retry budget exhausted after {attempt} attempts ({reason})"
            ) from last_exc
        log(f"attempt {attempt} failed ({reason}); sleeping {delay:.1f}s")
        time.sleep(delay)

Usage:

client = anthropic.Anthropic(max_retries=0, timeout=30.0)
 
reply = call_with_retries(lambda: client.messages.create(
    model="claude-sonnet-5-5",
    max_tokens=512,
    messages=[{"role": "user", "content": "Summarize this ticket in one sentence."}],
))

Notes on the code:

• retry-after is read as a number of seconds. If a provider sends a date form instead, the code falls back to backoff.
• Timeouts and connection failures are caught explicitly. The documentation names APITimeoutError for timeouts and APIConnectionError for connection failures, so test the wrapper against your SDK versions before relying on it.
• Request IDs come from the request-id header for Anthropic and x-request-id for OpenAI, which is what the SDK documentation describes.
• The wrapper does not make non-idempotent side effects safe. If the model call triggers a payment or an email, add your own idempotency key before retrying.

Reduce the chance of hitting limits

• Cache repeated input. For most Claude models only uncached input tokens count toward input-tokens-per-minute limits, so prompt caching raises effective throughput.
• Set max_tokens realistically on OpenAI. OpenAI says usage counts against TPM based on your max_tokens setting as well as input, so set it close to the expected response size. Anthropic states that max_tokens does not factor into its output-token rate limit calculations.
• Use batch APIs for offline work. OpenAI's Batch API runs large sets of requests without using your synchronous rate limits, and Anthropic offers the Message Batches API with its own limits.
• Watch the headers. Claude responses carry anthropic-ratelimit-requests-remaining, anthropic-ratelimit-input-tokens-remaining and related reset headers. OpenAI returns x-ratelimit-* fields. Log them to see headroom before a 429.

Verify your handling

1. Set a very low timeout such as 0.001 seconds on a test client and confirm you get an APITimeoutError and your wrapper logs the retries.
2. Point a client at a local stub that returns 429 with retry-after: 2 and confirm the wrapper sleeps about two seconds.
3. Return a 401 from the stub and confirm no retry happens.
4. Return a 429 whose body has error.code set to a non-retryable value and confirm it fails immediately.

Troubleshooting

Requests hang for minutes

You are on the 10-minute default timeout. Set timeout and use streaming for long outputs.

The SDK raises ValueError about expected duration

Anthropic's Python SDK raises it for non-streaming requests expected to run longer than about 10 minutes. Use streaming, the Batches API, or override timeout deliberately.

Retries never succeed on a 429

Check for the spend-cap case. If there is no retry-after and the error code is enforced_spend_limit_reached, or OpenAI returns a credit or spend-limit code, raise your limit or add credits instead.

Calls take far longer than your timeout

SDK retries multiply the delay. A 20-second timeout with 2 retries can reach about a minute. Lower max_retries or account for it in your deadline.

413 request too large

Anthropic lists maximum request sizes of 128 MB for the Messages API. Reduce the payload.

Checklist

• Set an explicit timeout on every client.
• Use streaming for long generations and Batches for offline work.
• Decide whether the SDK's 2 retries are enough before adding your own layer.
• Retry only transient errors, and honor retry-after.
• Never retry spend-cap or exhausted-credit 429s.
• Cap attempts and total retry time, and add jitter.
• Log the request ID of every failure.

References

• Claude API errors
• Claude API rate limits
• Anthropic Python SDK
• OpenAI rate limits guide
• OpenAI error codes
• OpenAI Python library