Circuit Breaker — AI Glossary

Reference details

Related terms

• Retry with Exponential Backoff
• Model Routing
• AI Gateway

How it works

1. Failures against a dependency are counted over a window, so the trigger is a rate rather than a single error.
2. Past a threshold the breaker opens and subsequent calls fail immediately without attempting the dependency.
3. That fast failure is the point: it stops the caller paying a full timeout per request while the dependency is down.
4. After a cooldown the breaker goes half-open and admits a limited number of probe requests to test recovery.
5. Probe success closes it and resumes normal traffic; probe failure reopens it, so recovery is tested rather than assumed.

Example: What fast failure saves during an outage

A dependency is down and each call times out after 10 seconds. At 100 requests per second with no breaker, 1,000 requests are simultaneously waiting at any moment, each holding a connection and a worker — which is how one dependency's outage exhausts the caller's own capacity. An open breaker fails those calls in microseconds, so the fallback path runs immediately and the caller stays healthy. The breaker protects the caller, not the dependency, which is the part most often misunderstood.

Commonly confused with

Confused with retry logic and with rate limiting, and all three sit in the request path for different reasons. A retry reissues an individual failed call, which is right for a transient blip and actively harmful during an outage because it multiplies load. A breaker stops calling entirely once failures are systemic. Rate limiting caps traffic by policy regardless of health. They compose in that order — retry within a breaker, both under a limit — and retrying without a breaker is the configuration that turns a dependency's outage into your own.

When to use it

Reach for it when:

• A dependency's failure would otherwise consume caller capacity in timeouts.
• A fallback exists that should run immediately rather than after a timeout.
• Recovery should be tested with probes rather than by resuming full traffic.

Reach for something else when:

• Failures are isolated and transient, where a bounded retry is the correct response.
• No fallback exists, so fast failure only moves the error earlier.
• Thresholds are untuned, where a breaker trips on normal variance and becomes an outage of its own.

Continue your research

Use these connected ModelRefs sections to compare alternatives, inspect implementation paths, and review the evidence and governance boundaries relevant to Circuit Breaker — AI Glossary.

• Browse models
• Review providers
• Inspect benchmarks
• Explore workflows
• Learn AI concepts
• Read decision guides
• Review trust and methodology
• Start a decision path