Idempotency

An operation is idempotent if performing it once or many times produces the same result.

• DELETE /users/42 is idempotent: the user is gone after the first call; subsequent calls do nothing new.
• POST /charge { amount: 100 } is not idempotent by default: each call charges 100 more dollars.
• PUT /users/42 { name: 'Alice' } is idempotent: setting the name to 'Alice' twice is the same as once.

This property matters because the network is unreliable. A client sends a request, doesn't get a response, retries. Without idempotency, the retry might charge the user twice. With idempotency, the duplicate is harmless. That's why every payment, every webhook, every "please don't fire twice" event needs this baked in.

Retries are everywhere:

• HTTP clients retry on timeouts and 5xx.
• Message queues retry on consumer failure.
• Mobile apps retry when network reconnects.
• Background workers retry after a crash.

At least one of these will fire a request that already succeeded but whose response was lost. Every operation that mutates state will be called twice eventually. You can either design for it (idempotency) or pay for it in incident reports.

The canonical example: Stripe's API has supported Idempotency-Key since 2015. Not by accident — they realized early that without it, network blips became double charges. Every modern payment, billing, and webhook API now offers something equivalent.

The standard pattern: the client supplies a unique key per logical request. The server stores (key → response). A retry with the same key returns the cached response without re-executing.

@app.post("/charge")
def charge(req, idempotency_key: str = Header(...)):
    if (cached := store.get(idempotency_key)):
        return cached.response
    with db.transaction():
        result = do_the_charge(req.amount, req.card_id)
        store.put(idempotency_key, request=req, response=result, ttl=24h)
    return result

Key rules:

• The key is client-generated; usually a UUID per logical attempt (not per HTTP retry).
• The key + the original request body should be stored so you can detect different requests with the same key (a sign of a client bug). Reject those with a 422.
• TTL is long enough for retries (24 hours is standard) but not forever (the dedupe store would grow without bound).

Sometimes you don't need a key — you can design the operation to be naturally idempotent.

• Use upserts, not inserts. INSERT ... ON CONFLICT DO UPDATE handles duplicates without ceremony.
• Use deterministic IDs. Instead of letting the database generate an order ID, accept one from the client. Duplicate POSTs with the same ID become a single row.
• Set, don't increment. SET balance = 100 is idempotent; balance += 10 is not. Where possible, use snapshot semantics.
• Use PUT, not POST, for resources whose identifier is known. PUT /users/42 is naturally idempotent by HTTP convention; POST /users is not.

When the API shape supports it, natural idempotency is cheaper than a dedupe layer. The dedupe layer is a fallback for when you can't redesign the operation.

Network drop after work? Retry is safe.

An operation does two things: write to your DB, then call an external API. After step 1 succeeds, the server crashes before step 2. On retry, you have to redo step 2 — but not step 1.

Approaches:

• Idempotent subcomponents: make each external call idempotent too (most modern APIs accept keys; pass yours through).
• Outbox pattern: write a row to an outbox table in the same DB transaction as your data write. A separate worker reads the outbox and makes the external call. If the worker crashes mid-call, it retries the same outbox row — and the external API's idempotency key (which you store in the row) makes the retry safe.
• Sagas: model long workflows as named steps with compensating actions. Each step is idempotent; failure triggers compensations.

The principle: turn one un-idempotent compound operation into a chain of idempotent atomic ones, with state stored at each boundary.

Real incidents teams report:

• Double charges: payment processor retried; both succeeded. Customer charged twice. Refund pipeline triggered.
• Duplicate orders: client's network blipped on submit; user tapped retry; two orders shipped.
• Spam: webhook delivery layer retried 14 times during a backend outage; recipient got 14 notification emails.
• Inventory miscounts: a decrement_stock call retried after a partial failure; inventory went negative.
• Replication storms: a sync job retried whole files instead of resuming, eating bandwidth and causing customer-visible slowdowns.

In every case the root cause was "this operation isn't idempotent". The fix in every case looks like one of the patterns above. None are exotic; what's exotic is teams that ship without thinking about retries.

Before merging any endpoint that changes state, ask:

1. What happens if this is called twice with the same input? If "two of whatever" is the wrong answer, you need idempotency.
2. How will the client supply an idempotency key, or can the operation be naturally idempotent?
3. What's the TTL on the dedupe store? 24 hours covers most retry scenarios; longer for batch systems.
4. What if the body differs between attempts with the same key? Return 422 — that's a client bug, not a retry.
5. Does the operation call external systems? Each external call needs its own idempotency story (key passthrough, outbox, saga).
6. Is it documented? Clients can't supply keys they don't know about.

Five minutes of design at write time saves five days of incident response when retries inevitably collide with reality.