Exponential backoff and jitter calculator

Pick a retry policy and set its base delay, cap and the maximum number of attempts; the schedule table gives the sleep before each retry and the longest a client can spend waiting. The scenario below it runs that policy for many clients at once: each needs one successful call, the server rejects everything while it is down, and afterwards it serves a fixed number of calls per millisecond and rejects the rest. The chart shows the calls arriving each millisecond, and the table on the right runs all six policies on the same outage.

• Times are in milliseconds and the simulation steps one millisecond at a time. "Rerun with new randomness" draws new jitter and start times.
• Copy answer gives the schedule and the comparison as text.

Many failures in distributed systems are brief: a dropped packet, a server restarting, a request that lost a race. Retrying turns these into successes. But when a service fails because it is overloaded, every retry is more load, and clients that retry at once can keep it down long after the original cause has gone.

A retry strategy decides how long to wait before each retry and when to stop. Waiting longer after each failure (backoff) gives the service room to recover; randomising the wait (jitter) stops clients that failed together from retrying together.

Exponential backoff doubles the wait after every failure, up to a cap: the sleep before retry k is min(cap, base × 2^k). The first retries come quickly, so a brief glitch costs little; a long outage quickly pushes clients to the cap. With a base of 100 ms and a cap of 5 seconds:

Backoff alone leaves clients synchronised: if a thousand requests fail in the same millisecond, they all retry 100 ms later, then 200 ms after that, in waves as big as the original burst. Marc Brooker’s 2015 AWS Architecture Blog post compared three ways of adding randomness:

• Full jitter: sleep a random time between 0 and min(cap, base × 2^k).
• Equal jitter: sleep half of that value plus a random time up to the other half, so never less than half.
• Decorrelated jitter: sleep a random time between the base and three times the previous sleep, capped.

In his simulation of clients contending to update the same database row, plain exponential backoff was the clear loser, doing more work and taking longer than any jittered version. Full and equal jitter made similar numbers of calls, but equal jitter took longer; between full and decorrelated jitter there was no clear winner.

This page’s simulation asks a different question, recovery from an outage: 500 clients start within 5 ms of each other, the server rejects every call for the first 50 ms, then serves 20 calls per millisecond. Each policy uses a base of 10 ms, a cap of 1,000 ms and up to 10 attempts. Averaged over eight runs:

Backoff limits how fast clients retry, not how much. Google’s Site Reliability Engineering book describes two budgets: at most three attempts per request, after which the failure is passed up to the caller, and a per-client limit that allows retries only while they are under 10% of requests. During an overload the second budget keeps the extra load to about 1.1 times normal, instead of the nearly threefold growth that three attempts per request would allow. The book also warns that retrying at several layers of a stack multiplies the attempts, so retries belong in one layer.

Some calls should not be retried at all. A request that is not idempotent, such as a payment without an idempotency key, may run twice. Errors that will not go away, like a malformed request or a permission failure, should fail fast. And a client that has passed its caller’s deadline should stop.