Timeouts, Retries, Idempotency

Timeouts, Retries, Idempotency are three settings that people configure separately and that only make sense together. A timeout decides when you give up, a retry decides what you do next, and an idempotency key decides whether doing it again is safe, so choosing any two without the third produces either lost work or duplicated work.

Key takeaways

• Derive the timeout from the dependency's measured latency distribution. A timeout below its p99 for successful calls guarantees you will fail at least one percent of healthy requests.
• Retries multiply load. Three attempts per layer across four layers is up to eighty-one calls at the leaf for one user request, which is how a slow dependency becomes an outage.
• Google's SRE overload chapter bounds this with a per-request limit of three attempts and a per-client retry budget holding retries under ten percent of requests, capping amplification near 1.1 times.
• Brooker's 2015 AWS post reports that full jitter, sleeping a random interval between zero and the capped exponential bound, reduced call count by more than half against exponential backoff without jitter.
• A retry is only safe where the receiver can prove it has seen the request before. Stripe's API replays the first result for a repeated key and rejects a reused key carrying different parameters.

Read this after Chapter 7, which handled work arriving faster than it can be finished, and before Chapter 9, which handles a whole dependency being gone rather than one call failing. The two chapters split the same Google SRE material deliberately: criticality and adaptive throttling live in Chapter 7, and the attempt limit and retry budget live here. The long-horizon reliability argument in the book on agents that finish applies the same three settings to work measured in hours rather than milliseconds.

A payment integration is configured with a thirty second timeout and three retries, which sounds cautious. It works for two years.

The provider has a slow afternoon. Response times move from 200 milliseconds to eight seconds, still succeeding. Every request now holds a worker for up to eight seconds, the pool saturates, the retries triple the offered load, and the checkout page starts timing out for everyone.

Then the reconciliation report arrives. Some customers were charged twice, because a timeout is not a failure, it is an unknown.

A timeout is derived, not chosen

A round number is not a timeout. It is a placeholder somebody left in.

The derivation needs one input: the measured latency distribution of the dependency for successful calls. Set the timeout above the percentile of that distribution you are willing to keep, and you have made an explicit choice about how many healthy calls you abandon. Set it below p99 and you have guaranteed that at least one percent of perfectly good requests fail, permanently, by configuration.

There is a ceiling too, and it comes from Chapter 1. If the enclosing request has a p99 budget of 300 milliseconds, a dependency timeout of thirty seconds is not cautious. It is incoherent, because ninety-nine times out of a hundred the user has left before the timeout fires.

So a timeout has two constraints: above the dependency's successful-call percentile, and below the caller's remaining budget. When those two ranges do not overlap, you have found a real design problem rather than a configuration one.

A retry is load multiplication

The dangerous property of a retry is that it is a local decision with a global effect, and the effect compounds per layer.

Do the arithmetic with round illustrative numbers. Four layers, each configured with three attempts, each unaware of the others. One user request becomes up to three calls at the second layer, nine at the third, twenty-seven at the fourth and eighty-one at the leaf. The leaf is usually the database.

That multiplier arrives at exactly the wrong moment. Retries fire when the dependency is already slow or failing, so the load multiplication is triggered by the condition it makes worse. This is the positive feedback loop Google's cascading failure chapter describes, and it is why a dependency that recovers on its own would have recovered faster if nobody had retried.

The rule that follows is uncomfortable and correct. Retry at one layer, not at every layer, and make the layer an explicit decision recorded in the design.

Jitter is the cheapest correction available

Backoff spreads retries out in time. Without jitter it spreads them into synchronised waves, because every client that failed at the same moment waits the same interval.

Brooker's AWS Architecture Blog post of 4 March 2015 measured the alternatives with 100 contending clients. Full jitter, sleeping for a random interval between zero and the capped exponential bound, reduced their call count by more than half compared with exponential backoff and no jitter. Decorrelated jitter, where the next sleep is a random value between the base and three times the previous sleep, capped, gave similar completion time with somewhat more work.

The formula is short enough to memorise. Sleep for a random amount between zero and the minimum of your cap and base times two to the attempt number. That single line removes the synchronised wave, and it costs nothing.

Do not run your own comparison to justify this. The measurement exists, it is dated, and it is first-party from somebody operating at the scale where it matters.

Three attempts, and a ten percent budget

Jitter changes the timing of retries. It does not bound their number, and bounding the number is the part that prevents an outage.

Google's overload chapter gives two limits worth adopting verbatim. A per-request limit of three attempts, so no single request can generate unbounded work. And a per-client retry budget that holds retries under ten percent of requests, which the chapter notes bounds worst-case amplification near 1.1 times.

Compare that against the unbudgeted arithmetic and the effect is stark. Eighty-one calls at the leaf becomes 1.1 to the fourth power, which is about 1.46. One user request produces roughly one and a half leaf calls instead of eighty-one, and the difference is one counter per client.

The chapter pairs this with the response shape from Chapter 7. A backend that is deliberately shedding should return overloaded and do not retry, because a retry against a server that is refusing on purpose is pure waste. Honour that signal in the client, or the budget is the only thing standing between you and the amplification.

Propagate the deadline, not the timeout

There is a subtler failure that timeouts alone cannot prevent, and it wastes capacity precisely when capacity is scarce.

Google's cascading failure chapter describes the fix as deadline propagation. Instead of each hop starting its own timeout, the caller passes an absolute deadline, and every stage checks it before starting work. A stage that finds the deadline already passed stops immediately rather than doing work nobody will read.

Without it, a request abandoned by the user at 300 milliseconds continues to consume database time at 2 seconds, 4 seconds and 8 seconds down the chain. During an incident that abandoned work can be most of the load, which means the system is spending its remaining capacity on results that will be discarded.

Propagation is a plumbing job and it has to be designed in early, because it needs a field on every internal call. Chapter 10 makes the same argument about identifiers, for the same reason: things that must appear on every hop cannot be retrofitted cheaply.

An idempotency key is what makes a retry safe

Everything above is about load. This section is about correctness, and it is the part that turns an incident into a data problem.

A timeout tells you that you did not get a response. It does not tell you whether the work happened. Retrying without a way for the receiver to recognise a repeat is a decision to perform the work twice whenever the response was lost rather than the request.

Stripe's API documentation describes the shipped version of the mechanism. The client sends an Idempotency-Key header. The first result, including its status code and body, is saved and replayed for later requests carrying the same key. Keys may be pruned after at least twenty-four hours. POST supports keys while GET and DELETE do not need them. And a request that reuses a key with different parameters is rejected with an error.

Read that last clause twice, because it is the part homegrown implementations omit. A key that is honoured for different parameters is not idempotency, it is a silent overwrite. And note the retention requirement: a key store with a five-minute window does not protect against the retry that happens after a queue drains an hour later.

There is an IETF effort in this space and it is worth naming accurately. The Idempotency-Key header draft in the httpapi working group, revision 07 dated 15 October 2025, is an expired Internet-Draft. It is not a standard. So treat the header name as a widely followed convention, and rest your design on your provider's documented behaviour.

At-least-once is the default you absorb

If you use a queue, duplication is not an edge case introduced by your retries. It is in the delivery contract.

Amazon's SQS documentation states that standard queues ensure at-least-once message delivery. Due to the highly distributed architecture, more than one copy of a message might be delivered, and messages may occasionally arrive out of order. It says standard queues suit applications that can handle duplicates and reordering.

That sentence is a requirement on your consumer, not a caveat about the queue. Every consumer of a standard queue needs either an idempotent operation or a dedupe key with a retention window longer than your maximum redelivery gap.

The same requirement appears inside a single database. PostgreSQL's documentation states that applications using the serialisable isolation level must be prepared to retry transactions after serialisation failures, which means retry logic exists even in a design with no network in it. Chapter 3 introduced that error. This is where you handle it.

A worked example, composited and labelled

The details here are composited from ordinary production shapes rather than one system, and the shape is exact.

A checkout flow calls a payment provider whose successful responses have a p99 of 1.2 seconds and a p99.9 of 3 seconds. The enclosing request is budgeted at p99 under 4 seconds.

The three numbers fall out. The timeout is 3.5 seconds, above the provider's p99.9 and inside the caller's budget. Retries are capped at two additional attempts, with full jitter and a base of 200 milliseconds. The client keeps a budget so retries never exceed ten percent of its requests. And the charge call carries an idempotency key derived from the order identifier plus the attempt-invariant amount, retained for seven days rather than for one.

The recorded constraint is that a lost response never produces a second charge, guaranteed by the key and its retention rather than by the retry policy. There are two reversal signals. The provider's p99.9 moving above 3 seconds for a week invalidates the timeout. Any duplicate charge in reconciliation invalidates the key design.

The objection: this is a lot of machinery for a rare event

The objection is fair on frequency and wrong on cost, which is the usual pattern for reliability work.

The events are rare and their cost is asymmetric. A duplicate charge is a refund, a support conversation, a trust loss and sometimes a chargeback fee. It also arrives in batches, because the condition that caused it affected many requests at once. A retry storm is an outage rather than a slowdown, because it converts a degraded dependency into a saturated one.

The machinery is also smaller than it sounds. A derived timeout is one measurement and one number. Jitter is one line. The budget is one counter per client. The key is one column with a unique constraint and a stored response. Four small things, written once, applied by convention.

What is genuinely expensive is discovering you need them during the incident, then writing a reconciliation script against production data with somebody watching.

Chapter summary

Timeouts, retries and idempotency are one decision in three parts. A timeout is derived from the dependency's measured distribution for successful calls, above the percentile you intend to keep and below the caller's remaining budget, so a value under the dependency's p99 guarantees failing one percent of healthy calls. Retries multiply load at the moment the system can least afford it, since three attempts across four layers is up to eighty-one leaf calls per user request, which is the positive feedback Google's cascading failure chapter describes. Brooker's 4 March 2015 AWS post measured full jitter reducing call count by more than half against undithered exponential backoff with 100 contending clients, and Google's overload chapter bounds the count with a three-attempt per-request limit and a per-client budget keeping retries under ten percent of requests, holding worst-case amplification near 1.1 times, which turns eighty-one leaf calls into about 1.46. Deadline propagation stops stages working on requests the caller has abandoned, and like trace identifiers it must be designed in because it needs a field on every internal call. Correctness then depends on the receiver recognising a repeat: Stripe's documentation replays the saved first result for a repeated key, retains keys for at least twenty-four hours and rejects a reused key with different parameters, while SQS standard queues state at-least-once delivery with possible duplicates and reordering. The chapter's decision is three recorded numbers plus a key design, with the dependency percentile that reverses them.

One call failing is now handled. The harder question is what happens when there is nothing on the other side at all, for four minutes or forty, and how many of your customers that is allowed to reach. Chapter 9 is Failure Domains and the Blast Radius You Chose.