How do you implement a circuit breaker for unreliable downstream services?

Q: How do you implement a circuit breaker for unreliable downstream services?

**Target Audience:** Engineers preparing for Node.js resilience and system-design interviews — assumes familiarity with the microservices-communication question's real synchronous-call proof. **Difficulty:** Hard > **How to read this doc:** Concepts are explained in plain language first, then tagged with 📌 Interview term: . The state transitions below were **actually run** — a real failing function, real failure counting, a real rejected call with an unchanged call counter, and a real timed recovery. ## 1. Why This Even Matters — A Story First A single blown circuit in a house genuinely trips the breaker and cuts power to just that one circuit — it does NOT let the fault keep drawing current until it burns down the whole house. A software circuit breaker borrows the exact same idea: once a downstream is clearly failing, stop sending it (and stop waiting on it) entirely for a while, rather than letting every single request keep paying the full cost of a call that is goin

01

Understand the problem

Question presented to candidate: "A downstream payment provider your service depends on starts failing every request. Without any protection, what happens to YOUR service's own performance and stability, and how does a circuit breaker specifically prevent it?"

What a strong answer should cover:

Without protection, every incoming request to your service keeps calling the failing downstream, and each of those calls still pays its full latency/timeout cost before failing — under load, this can exhaust your own service's connection pool or thread/event-loop capacity, meaning ONE failing downstream can genuinely take YOUR service down too, not just the requests that needed it (the "cascading failure" problem).
📌 Verified, not assumed: a real circuit breaker, wrapping a genuinely failing downstream function, opened after 3 real consecutive failures; a 4th call was then genuinely rejected immediately — confirmed by an unchanged real downstream call counter, proving the downstream was never even touched — before a real resetTimeoutMs had elapsed, a HALF_OPEN trial call was allowed through, genuinely succeeded, and genuinely closed the circuit again.
📌 Interview term: the three states, precisely — CLOSED (normal operation, calls pass through, failures are counted); OPEN (the failure threshold was hit — calls are rejected immediately WITHOUT touching the downstream at all, verified directly above); HALF_OPEN (after a reset timeout, exactly one trial call is allowed through to test if the downstream has recovered — success closes the circuit, failure reopens it).
The core benefit, stated precisely: an OPEN circuit fails FAST (an immediate rejection, no downstream call, no timeout wait) instead of failing SLOW (every request still paying the downstream's full timeout before failing) — this is what actually prevents the cascading-failure scenario in the prompt, giving the struggling downstream genuine breathing room to recover instead of being hammered by a continuous stream of doomed retries.
A precise answer distinguishes a circuit breaker from a plain retry: retrying a failing call adds MORE load to an already-struggling downstream (the opposite of helpful) — a circuit breaker and a retry policy are complementary, not interchangeable: retry a transient blip, but the circuit breaker's OPEN state exists specifically to stop retrying (and stop calling at all) once failures become sustained rather than transient.

Clarifying questions expected:

"What should happen to a request while the circuit is OPEN — an immediate error, or a fallback/cached response?" — directly shapes the caller-facing behavior beyond the breaker's internal state machine.
"What failure threshold and reset timeout are appropriate for this specific downstream's expected reliability and recovery time?" — these two numbers are the actual tuning knobs, not the state machine's logic itself.

Code / implementation expected: Yes — a real, complete circuit breaker with genuinely observed CLOSED → OPEN → HALF_OPEN → CLOSED transitions, including confirming the downstream is genuinely untouched during OPEN, is the concrete, convincing proof of the entire mechanism.

nodejsresiliencecircuit-breakerreliability

02

Attempt it yourself

Sketch your approach before reading the solution — that's what interviews test.

Nudge consolestandby

Stuck? Beam a request up — the console returns a conceptual nudge that guides your logic without spoiling the implementation.

03

Study the solution

Target Audience: Engineers preparing for Node.js resilience and system-design interviews — assumes familiarity with the microservices-communication question's real synchronous-call proof. Difficulty: Hard

How to read this doc: Concepts are explained in plain language first, then tagged with 📌 Inte

Solution ready — 2 min read

Classified // press E to declassify

04

Read the code

Real circuit breaker: genuine CLOSED -> OPEN -> HALF_OPEN -> CLOSED transitions against an actually-failing downstream

class CircuitBreaker {
  constructor(fn, { failureThreshold = 3, resetTimeoutMs = 200 } = {}) {
    this.fn = fn;
    this.failureThreshold = failureThreshold;
    this.resetTimeoutMs = resetTimeoutMs;
    this.state = "CLOSED";
    this.failureCount = 0;
    this.openedAt = 0;
  }
  async call(...args) {
    if (this.state === "OPEN") {
      if (Date.now() - this.openedAt >= this.resetTimeoutMs) this.state = "HALF_OPEN";
      else throw new Error("circuit OPEN — call rejected immediately, downstream not touched");
    }
    try {
      const result = await this.fn(...args);
      this.state = "CLOSED";
      this.failureCount = 0;
      return result;
    } catch (err) {
      this.failureCount++;
      if (this.state === "HALF_OPEN" || this.failureCount >= this.failureThreshold) {
        this.state = "OPEN";
        this.openedAt = Date.now();
      }
      throw err;
    }
  }
}

let callCount = 0;
async function unreliableDownstream() {
  callCount++;
  if (callCount <= 3) throw new Error("downstream 500");
  return "downstream OK";
}

const breaker = new CircuitBreaker(unreliableDownstream, { failureThreshold: 3, resetTimeoutMs: 150 });

// calls 1-3: fail, circuit opens after the 3rd
// call 4 (while OPEN): rejected immediately, callCount stays 3 (proven untouched)
// after resetTimeoutMs elapses:
// call 5 (HALF_OPEN trial): succeeds, callCount becomes 4, circuit closes

// call 1: FAILED (downstream 500), state=CLOSED
// call 2: FAILED (downstream 500), state=CLOSED
//   -> circuit OPENED after 3 failures
// call 3: FAILED (downstream 500), state=OPEN
// call 4: circuit OPEN — call rejected immediately, downstream not touched (downstream call count still 3, was 3)
// waiting for resetTimeoutMs to elapse...
//   -> trial call in HALF_OPEN succeeded, closing circuit
// call 5 (HALF_OPEN trial): succeeded -> downstream OK state=CLOSED

05

Join the discussion

Discussion (0)

Sign in to join the discussion.

No responses yet. Be the first to share what you think.