# Idempotent API Design: Keys, Storage and Retention for Payment-Style Calls

> Design POST endpoints that clients can safely retry: the Idempotency-Key header, a PostgreSQL key table, request fingerprints, concurrent retries, status codes and how long to keep keys.

Source: https://www.epifive.com/blog/idempotent-api-design-keys-storage-retention · Published 2026-10-25 · EpiFive

## Key takeaways

- An idempotency key lets a client retry a non-idempotent POST safely, because the server returns the stored result of the first attempt instead of repeating the side effect.
- Store the key, a fingerprint of the request, the status and the response, and make the key unique so concurrent duplicates cannot both run.
- Reject a reused key with a different payload (422 in the IETF draft) and return 409 when the original request is still in progress.
- Publish how long you retain keys; Stripe's v1 API allows removal after 24 hours, and the right window depends on how long your clients retry.

To make a payment-style call safe to retry, require the client to send a unique `Idempotency-Key` header, store the key together with a fingerprint of the request and the result of the first attempt, and replay that stored result whenever the same key and the same request arrive again. A key reused with a different request is an error, a duplicate that arrives while the first is still running gets a conflict, and keys expire after a documented retention period.

## Who this is for

This is for API designers and back-end developers who expose operations with side effects, such as creating a charge, placing an order or sending a message, to clients that retry on timeouts. By the end you will have a header contract, a PostgreSQL table, a request flow that handles concurrency and a retention policy.

## Why retries are dangerous without a key

HTTP's safe retries are limited to idempotent methods. `GET`, `PUT` and `DELETE` are idempotent by definition: repeating them leaves the server in the same state. `POST` and `PATCH` are not. If a client sends `POST /payments` and the connection drops before it sees a response, it cannot tell whether the charge happened. Retrying risks charging twice; not retrying risks never charging.

Stripe's API reference describes the remedy: use an idempotency key when creating or updating an object, so that after a connection error you can repeat the request without risk of creating a second object or performing the update twice. The IETF's httpapi working group has an active draft, `draft-ietf-httpapi-idempotency-key-header`, that standardises the header. It states that the aim is to let clients safely retry non-idempotent methods such as `POST` and `PATCH`, so a client that times out can tell whether the server already acted. Because it is still a draft, treat the details as current direction rather than a final standard.

## The header contract

Define the contract in your API documentation before writing code.

- **Header name:** `Idempotency-Key`. In the IETF draft it is an Item Structured Field whose value must be a string.
- **Who generates it:** the client. The draft says the resource owner defines what makes a key unique, that clients should generate it, and that UUIDs are recommended. Stripe suggests V4 UUIDs or another random string with enough entropy, allows keys up to 255 characters and advises against sensitive data such as email addresses in keys.
- **Which operations require it:** all non-idempotent writes that move money or create resources. If a required key is missing, the draft says to answer `400 Bad Request`, ideally with a link to documentation, optionally using RFC 9457 problem details.
- **Scope:** keys should be unique per client. The draft warns that low-entropy keys could let attackers guess other clients' keys, and suggests a composite lookup key that includes client-specific data. Always scope the stored key by authenticated client or tenant.

A request looks like this:

```http
POST /v1/payments HTTP/1.1
Host: api.contoso.com
Authorization: Bearer <token>
Idempotency-Key: 8e03978e-40d5-43e8-bc93-6894a57f9324
Content-Type: application/json

{"amount": 4200, "currency": "usd", "customer": "cus_123"}
```

## Server behaviour

The draft describes the behaviours a server should implement.

| Situation | Response |
| --- | --- |
| First request with this key | Process normally and store the outcome |
| Retry after the first request finished | Respond consistently with the original outcome |
| Retry while the first request is still running | `409 Conflict` |
| Same key, different payload | `422 Unprocessable Content` |
| Required key missing | `400 Bad Request` |

Stripe's v1 behaviour is a useful reference. It saves the status code and body of the first request made with a key once endpoint execution begins, and retries with the same key return the saved response, including `500` errors. It compares incoming parameters with those of the original request and returns an error if they differ. It does not save a result if the parameters fail validation or the request conflicts with a concurrently executing one, because no endpoint execution started; those requests can be retried.

You must decide how your API treats failures. Replaying a stored `500` is consistent, but it can lock a client into a transient error. A common refinement is to store the result only for outcomes that are final (success or a definitive business error) and let the client retry after server errors with the same key, provided your processing is itself safe to repeat. Whichever you choose, document it.

## Step 1: Create the key table

Store keys in the same database as the business data so that recording the key and performing the operation can share a transaction. A PostgreSQL design:

```sql
CREATE TABLE idempotency_keys (
    client_id        text        NOT NULL,
    idempotency_key  text        NOT NULL,
    request_method   text        NOT NULL,
    request_path     text        NOT NULL,
    request_hash     bytea       NOT NULL,   -- fingerprint of the request body
    status           text        NOT NULL,   -- 'started' | 'completed'
    response_status  integer,
    response_body    jsonb,
    created_at       timestamptz NOT NULL DEFAULT now(),
    locked_at        timestamptz,
    PRIMARY KEY (client_id, idempotency_key)
);

CREATE INDEX idempotency_keys_created_idx ON idempotency_keys (created_at);
```

The composite primary key does three jobs: it scopes keys to a client, it makes duplicate inserts fail, and it provides the lookup. The `request_hash` is the fingerprint the IETF draft mentions: a server may derive a checksum or signature from the request payload and use it with the key. Hash the method, path and canonicalised body, for example with SHA-256.

## Step 2: Claim the key before doing the work

The critical move is to insert the row first, atomically, so that two concurrent requests with the same key cannot both proceed.

```sql
INSERT INTO idempotency_keys
    (client_id, idempotency_key, request_method, request_path, request_hash, status, locked_at)
VALUES
    (@clientId, @key, 'POST', '/v1/payments', @hash, 'started', now())
ON CONFLICT (client_id, idempotency_key) DO NOTHING
RETURNING 1;
```

- If the statement returns a row, you own the key. Run the operation.
- If it returns nothing, a row already exists. Read it and branch:

```sql
SELECT request_hash, status, response_status, response_body
FROM idempotency_keys
WHERE client_id = @clientId AND idempotency_key = @key;
```

Then apply the rules:

1. `request_hash` differs from the new request's hash: return `422`.
2. `status = 'completed'`: return the stored `response_status` and `response_body`.
3. `status = 'started'`: return `409`, optionally with a `Retry-After` header.

## Step 3: Execute and record the result atomically

Do the business write and mark the key completed in the same transaction wherever you can, so a crash cannot leave a completed payment with a key stuck in `started`.

```sql
BEGIN;

INSERT INTO payments (payment_id, customer_id, amount, currency)
VALUES (@paymentId, @customerId, @amount, @currency);

UPDATE idempotency_keys
SET status = 'completed',
    response_status = 201,
    response_body = @responseJson
WHERE client_id = @clientId AND idempotency_key = @key;

COMMIT;
```

When the operation includes a call to an external system such as a card processor, the single-transaction approach no longer covers everything. Pass your own idempotency key, or a value derived from it, to the downstream call, so a retry of your handler does not charge twice downstream. Microsoft's write-through caching architecture follows the same idea for command-style writes: it validates the request, checks the idempotency key, commits to the database and records follow-up work in an outbox, and it recommends that both the write function and the repair function be idempotent. See [Transactional Outbox Pattern](https://www.epifive.com/blog/transactional-outbox-pattern-reliable-events) for the follow-up-work technique.

## Step 4: Recover stuck keys

A server can crash after claiming a key and before finishing. Without a recovery rule the client receives `409` until the row is removed. Use `locked_at` to detect it: if a `started` row is older than your maximum processing time, a new request may take it over after checking whether the work already happened, for example by looking for a payment row that references the key. Choose a lock timeout comfortably longer than your slowest legitimate request and test the takeover path explicitly.

## Step 5: Set and publish a retention policy

Keys cannot live forever, and the table would grow without bound. The draft says servers may purge keys on a schedule and should publish that policy. Stripe's v1 API documents that keys can be removed automatically once they are at least 24 hours old, after which reusing a key creates a new request.

Choose retention from the longest realistic client retry window, including queued or batch clients that retry hours later, and then document it. A purge job is simple:

```sql
DELETE FROM idempotency_keys
WHERE created_at < now() - interval '24 hours';
```

Delete in batches on large tables, and keep the retention at least as long as the longest retry window you promise clients. If the unique key in your payment table also guards against duplicates, you retain a safety net after the key row has been purged.

## Client guidance to publish

- Generate a new key per logical operation, not per attempt. Reuse the same key on every retry of that operation.
- Never reuse a key for a different request.
- Retry with backoff on timeouts, connection errors and `5xx` responses, and stop after a bounded number of attempts.
- Treat `409` as "the first attempt is still running": wait and retry with the same key.
- Treat `422` as a client bug: the key was reused with a different body.

## Troubleshooting

**Clients get `409` for a long time.** A key is stuck in `started`. Check `locked_at`, the lock timeout and whether the request crashed mid-flight.

**Duplicate charges still happen.** The downstream call is not idempotent, or the key is generated per attempt on the client. Pass a derived idempotency key downstream, and verify the client reuses one key across retries.

**`422` errors after a client release.** The client now sends a different body with an old key, for example because a timestamp field changes per attempt. Remove volatile fields from the fingerprint or from the request.

**The table grows quickly.** Check that the purge job runs, and that retention matches the policy you published.

**Two tenants collide.** The key is not scoped by client. Keep `client_id` in the primary key.

## Checklist

- `Idempotency-Key` required on non-idempotent writes, with `400` when missing.
- Keys scoped to the authenticated client.
- Unique constraint on the key; insert before work.
- Request fingerprint stored and compared, `422` on mismatch.
- `409` for a duplicate that arrives while the first runs.
- Result stored in the same transaction as the business write.
- Downstream calls carry their own idempotency key.
- Retention documented and enforced by a purge job.

To see the same problem from the receiving side of webhooks, read [Webhooks that don't double-charge](https://www.epifive.com/blog/webhook-retry-idempotency-patterns).

## References

- Stripe API reference, Idempotent requests: https://docs.stripe.com/api/idempotent_requests
- IETF, The Idempotency-Key HTTP Header Field (Internet-Draft): https://datatracker.ietf.org/doc/draft-ietf-httpapi-idempotency-key-header/
- Microsoft Learn, Write-through caching with Azure Managed Redis and Azure SQL Database: https://learn.microsoft.com/en-us/azure/architecture/databases/architecture/write-through-caching-azure-sql-managed-redis
- PostgreSQL documentation, INSERT (ON CONFLICT): https://www.postgresql.org/docs/current/sql-insert.html

## Frequently asked questions

### What is an idempotency key?

It is a unique value the client generates and sends with a request, usually in an Idempotency-Key header. The server uses it to recognise a retry of the same request and return the original outcome instead of performing the operation a second time.

### Which HTTP methods need idempotency keys?

POST and PATCH, because they are not idempotent by definition. GET, PUT and DELETE are idempotent by definition. Stripe, for example, accepts idempotency keys on POST requests and says sending them on GET and DELETE has no effect.

### What should the server do if the same key arrives with a different body?

Reject it. The IETF draft for the Idempotency-Key header recommends 422 Unprocessable Content for a key reused with a different payload, and Stripe returns an error when parameters differ from the original request.

### How long should I keep idempotency keys?

Long enough to cover the longest realistic client retry window. Stripe's v1 API allows keys to be removed once they are at least 24 hours old. Publish your retention period so clients know how long a retry is safe.