Why your Envoy circuit breaker never trips

The symptom

An upstream service saturates under load. Latency climbs, queues build, and the failure propagates back through callers. The circuit breaker that was configured specifically to contain this does not appear to activate, and the overflow counters stay at zero.

The usual conclusion is that the breaker is misconfigured. More often it is configured correctly and simply never reached.

Circuit breakers are per proxy, not per service

Envoy applies circuit breaker thresholds per cluster, per priority, within a single Envoy instance. In a mesh, every client workload has its own sidecar, and each sidecar enforces the threshold independently against its own view of the upstream.

The practical consequence is that the effective limit against the upstream is the configured threshold multiplied by the number of client proxies. A `maxConnections` of 100 with 50 client pods permits up to 5,000 connections in aggregate. Teams reason about the number as a service-wide budget, but it is not one — and this gap widens every time the client deployment scales up.

This is also why the same configuration behaves differently in staging and production. The threshold did not change; the number of enforcing proxies did.

Istio does not inherit the Envoy defaults

Envoy ships defaults for its circuit breaker thresholds — 1024 for max connections, pending requests, and requests, and 3 for retries. Reading only the Envoy documentation gives the impression that a modest bound exists even when nobody has configured anything.

Istio replaces these with values high enough to be effectively unlimited unless a `DestinationRule` sets them. A mesh with no explicit `connectionPool` configuration therefore has no meaningful connection or request bound at all, which is the opposite of the assumption most people start from.

Which threshold actually applies depends on the protocol

The settings are not interchangeable, and the one that matters depends on how the connection is negotiated.

• Over HTTP/1.1, each concurrent request needs its own connection, so the connection ceiling is the effective limit and pending requests queue behind it.
• Over HTTP/2, requests are multiplexed over a single connection, so a connection limit constrains almost nothing. The concurrent request ceiling is the one that binds.
• Retries have their own budget. Because a retry is issued by the client proxy, retry limits interact with concurrency limits — a retry storm can exhaust the request budget without the origin request rate changing at all.

How to confirm it

Envoy increments distinct counters for each overflow class, which is what makes this diagnosable rather than a matter of opinion. Read them per cluster on the client proxy, not on the upstream.

• Connection overflow and pending-request overflow counters tell you whether a limit was reached at all. Zero across the fleet during a saturation event means the breaker is not the mechanism doing anything.
• Requests rejected by a circuit breaker are returned as 503 and flagged as upstream overflow in the access log. That flag distinguishes breaker rejection from an upstream that actually returned an error.
• Compare observed per-pod concurrency against the configured threshold. If peak per-instance concurrency is an order of magnitude below the limit, the breaker cannot engage no matter how saturated the upstream is in aggregate.

What to change

Set thresholds from measured per-instance concurrency rather than from a service-wide capacity estimate divided by nothing. Establish what a single client pod actually drives at peak, set the limit above that but below the point where the upstream degrades, and re-check it when either deployment is resized.

Treat circuit breaking and outlier detection as separate tools, because they solve different problems. Circuit breakers bound concurrency against an upstream cluster. Outlier detection removes individual hosts that are behaving badly. A saturated but uniformly healthy upstream will not trigger outlier detection, and a single failing host will not be contained by a concurrency limit.