Idempotency Keys: Making API Retries Safe

You tap “Pay”, the spinner spins, the network coughs, and you get… nothing. No confirmation, no error, just a void. So you tap again. Did you just pay twice? On the internet, this is not a hypothetical — it is Tuesday. Networks fail in the most inconvenient way possible: not by clearly saying “no”, but by going silent right after they said “yes” to the server but before the “yes” reached you.

This is the double-charge problem, and the elegant fix is a single HTTP header: the idempotency key.

What “idempotent” actually means

In maths, an idempotent operation is one you can apply many times and get the same result as applying it once. Multiplying by one, taking the absolute value, pressing a lift button forty times — same outcome. In HTTP, some methods get this for free. GET reads and changes nothing. PUT and DELETE are idempotent by design: replacing a resource with the same payload, or deleting an already-deleted thing, lands you in the same final state.

POST is the troublemaker. “Create a charge” is not naturally safe to repeat — run it twice and you get two charges. So we bolt idempotency on from the outside.

The contract: one key, one outcome

An idempotency key is a unique, client-generated token attached to a request. The deal between client and server is simple:

The same key always produces the same result, no matter how many times you send it.

The client generates the key — and this matters. The server cannot tell a retry apart from a genuinely new “charge $20” request without a correlation token the client supplies. Use a V4 UUID or any random string with enough entropy that two real requests will never collide.

POST /v1/charges HTTP/1.1
Host: api.example.com
Idempotency-Key: 5f3a1c9e-7b2d-4e8a-9c1f-2a6b8d4e1f00
Content-Type: application/json

{ "amount": 2000, "currency": "usd", "customer": "cus_123" }

On the server, the logic is a cache keyed by that header:

def create_charge(key, params):
    saved = store.get(key)
    if saved is not None:
        # Replay the original outcome — status code AND body.
        return saved.status, saved.body

    status, body = process_payment(params)   # the real, dangerous bit
    store.set(key, Saved(status, body), ttl=24 * 3600)
    return status, body

The crucial detail: Stripe and friends save the result of the first request — success or failure — and replay it. If the original returned a 500, the retry gets that same 500, not a fresh attempt. The key locks in the outcome, not just the happy path.

The traps nobody warns you about

Same key, different payload. A client retries with the same key but a sneakily different body — $20 the first time, $2000 the second. Don’t silently honour either. Store a fingerprint of the request parameters alongside the key and reject mismatches with a 422. The key promises the same request, not any request.

The race condition. Two retries arrive within milliseconds, both miss the cache, both charge. Wrap the read-process-write in a lock or a unique database constraint on the key column so the second one blocks or fails cleanly instead of double-charging.

Retention. Keep keys long enough to cover real-world retries — a flaky mobile user reopening the app an hour later still counts. For retail payments, 24 hours is the widely cited sweet spot. Too short and a legitimate retry sails past your cache and charges again.

Naive retries. The client’s half of the bargain is retrying politely. Hammering a struggling server with instant retries causes a thundering herd that keeps it down. Use exponential backoff (wait roughly 2ⁿ seconds after the nth failure) plus random jitter so a thousand clients don’t all retry on the same tick.

import random, time

def with_retries(send, key, attempts=5):
    for n in range(attempts):
        ok, resp = send(key)          # SAME key every attempt
        if ok or not resp.retryable:
            return resp
        time.sleep(min(2 ** n + random.random(), 30))
    raise RetriesExhausted()

Note the same key on every attempt — that is the whole point. The key makes the retry safe; the backoff makes it kind.

Your idempotency checklist

Before you ship a mutating endpoint, run through this:

• Client generates the key (UUID v4), sends it as Idempotency-Key.
• Server caches the first response — status code and body — and replays it.
• Fingerprint the payload; reject same-key-different-body with a 422.
• Lock or unique-constraint the key to kill the concurrent-retry race.
• Set a TTL that matches your retry window (24h is a fine default).
• Client retries with exponential backoff and jitter, reusing the key.

Do this, and “the network died mid-payment” stops being a 3am incident and becomes a non-event. The same tap, twice, lands once — which is exactly what your users assumed was happening all along.