Circuit breaker simulator

The simulation runs one minute of traffic: requests arrive at random at the chosen rate, each is handled by one of 20 workers that calls a dependency, and a request that finds all workers busy is rejected. From second 15 to 35 the dependency is down, either refusing connections at once or hanging until the caller’s timeout. The same arrivals are run twice, without a breaker and with one, so the table compares like with like. The chart shows one run at a time, with the breaker’s state along the bottom.

Things to try:

• With Hangs until timeout, chart the run Without the breaker: the workers fill up and requests are turned away for the whole outage.
• Change the timeout from 500 ms to 5 s and watch how late the breaker opens.
• Switch to Fails fast with errors: without a breaker the caller survives, but keeps hammering the failing service.
• Follow the half-open trials in the log: some fail while the outage lasts, and the first set after it ends closes the breaker.

A circuit breaker sits between a service and something it depends on, such as another service, a database or an external API, and watches the outcomes of calls. When failures pass a threshold, it trips: for a while, calls fail immediately inside the caller, without being sent. Then it lets a few calls through to see whether the dependency has recovered.

The name comes from the electrical device that cuts a circuit when too much current flows, protecting the wiring. Michael Nygard described the software version in Release It! (2007) as a defence against cascading failures, where one slow component drags down everything that calls it, and Martin Fowler’s 2014 article made it widely known. Netflix’s Hystrix library brought it into common use in Java microservices.

• Closed. Calls go through normally. The breaker records each outcome in a sliding window, here the last 20 calls. Once the window holds at least the minimum number of calls (10), the breaker opens if the failure rate reaches the threshold (50%). Calls slower than a slow-call threshold can count too.
• Open. Every call is rejected at once (Resilience4j throws CallNotPermittedException), so the caller can return an error or a fallback in microseconds. Nothing is sent to the dependency.
• Half-open. After a wait (5 s here), the next call moves the breaker to half-open, and a fixed number of trial calls (5 here) are let through; further calls are rejected until the trials finish. If the trials’ failure rate is below the threshold the breaker closes and starts a fresh window; otherwise it opens for another wait.

These are the rules of Resilience4j’s count-based breaker, which the simulator follows; other libraries differ in detail, for example counting consecutive failures instead of a rate, or using a time-based window.

A failing dependency hurts its callers most when it fails slowly. Every call to a hung service holds a thread, a connection and memory until it times out. With 20 workers, a 2-second timeout and 50 requests a second, the workers are all stuck within a fraction of a second, and from then on the service rejects everything, including requests that never needed the broken dependency. That is how one failure cascades up a call chain. In the current settings, without a breaker 388 worker-seconds were spent waiting on the dead dependency; with one, 78.

Opening the breaker turns slow failures into instant ones. Workers are freed for requests that can still succeed, users get an answer (perhaps a degraded one) immediately, and the struggling dependency stops receiving traffic it cannot handle, which often is what it needs to recover. Little’s law makes the arithmetic plain: the number of requests in flight equals the arrival rate times the time each one takes, so cutting the time per failed request from seconds to microseconds empties the queue.

A circuit breaker can only count a failure when a call ends. For a dependency that hangs, that is when the caller’s timeout fires, so the breaker cannot react sooner than the timeout plus enough calls to fill its minimum. In the model, with the dependency hanging at 50 requests a second:

The simulator uses small numbers so that things happen within a minute. Resilience4j’s documented defaults are much more conservative:

A breaker is one of several tools that work together:

• Retries handle brief glitches, but during an outage they multiply load: three retries per request is four times the traffic to a service that is already failing. Use exponential backoff with jitter, cap the total, and put the breaker outside the retries, or count retries as failures, so an open breaker stops them.
• Fallbacks decide what the user gets while the breaker is open: a cached value, a default, a reduced page, or a clear error. A fallback that calls another remote service needs its own protection.
• Bulkheads give each dependency its own pool of workers or connections, so a hung dependency can only exhaust its own pool. In the simulator all requests share 20 workers; with bulkheads, requests that did not need the failing service would have been untouched.
• Load shedding and rate limiting protect a service from too much traffic in general, whereas a breaker reacts to a dependency’s failures.

• Hystrix, Netflix’s Java library, made circuit breakers, bulkheads and fallbacks standard in microservices. Netflix moved it to maintenance mode in 2018 and pointed new projects to Resilience4j.
• Resilience4j is a lightweight Java library with circuit breakers (count- or time-based windows, as described above), retries, bulkheads, rate limiters and time limiters that can be combined.
• Polly provides the same patterns for .NET.
• Envoy, the proxy under Istio and many service meshes, uses “circuit breaking” for limits on concurrent connections, pending requests, requests and retries per upstream cluster, which acts like a bulkhead. The open-and-wait behaviour of the classic pattern is closer to Envoy’s outlier detection, which ejects individual hosts that return consecutive errors and readmits them after an ejection time.

The difference matters in practice: a breaker in the client treats the dependency as a whole, while outlier detection in a load balancer removes only the bad instances and keeps sending traffic to healthy ones.

A breaker adds state and a new failure mode: an open breaker rejects requests that might have succeeded. It is not worth it for calls to local resources that cannot fail independently, for work that must eventually happen (queue it instead and retry later), or for dependencies with so little traffic that the window never fills. And it is no substitute for timeouts, capacity planning or fixing the dependency.

Circuit breakers come up in system design interviews about microservices and reliability: “what happens to your service when the payment provider is slow?” A good answer names the three states and what moves the breaker between them, explains why slow failures are the dangerous ones (exhausted threads and connections), puts timeouts and retries with backoff alongside the breaker, and says what the user sees while it is open. Mentioning bulkheads and where the breaker runs (in the client library or in a sidecar proxy) shows depth.