API Gateway Circuit Breaker — Gateway-Level Resilience for Microservices

In this tutorial, you will learn about API Gateway Circuit Breaker. We cover key concepts, practical examples, and best practices to help you master this topic.

API gateway circuit breakers provide centralized resilience for microservices by wrapping each route with per-service circuit breakers, returning gateway-level fallback responses, and coordinating circuit breaker state propagation across distributed services.

flowchart LR
    Client[Client] --> GW[API Gateway]
    GW --> CB1[CB: Payment Service]
    GW --> CB2[CB: Inventory Service]
    GW --> CB3[CB: Notification Service]
    CB1 -->|Open| Fallback1[Payment Fallback]
    CB2 -->|Open| Fallback2[Inventory Fallback]
    CB3 -->|Open| Fallback3[Notification Fallback]
    CB1 -->|Closed| P[Payment Service]
    CB2 -->|Closed| I[Inventory Service]
    CB3 -->|Closed| N[Notification Service]

What You'll Learn

• Per-route circuit breaker configuration
• Gateway-level fallback responses
• Circuit breaker state propagation
• Rate limiting with circuit breakers
• Gateway patterns: Spring Cloud Gateway, Kong, Envoy

Why It Matters

Without gateway circuit breakers, each microservice must implement its own resilience. Centralized circuit breakers at the gateway provide consistent fallback behavior across all services, reduce individual service complexity, and enable global resilience policies.

Real-World Use

DodaTech's Kong API gateway uses circuit breakers for all upstream services. When the payment service fails, the gateway returns a 503 with a JSON fallback instead of forwarding the request. This reduced payment service load by 70% during the last outage.

Spring Cloud Gateway Circuit Breaker

import org.springframework.cloud.gateway.route.RouteLocator;
import org.springframework.cloud.gateway.route.builder.RouteLocatorBuilder;
import org.springframework.context.annotation.Bean;
import org.springframework.context.annotation.Configuration;
import org.springframework.http.HttpStatus;
import org.springframework.web.server.ResponseStatusException;

@Configuration
public class GatewayCircuitBreakerConfig {
    @Bean
    public RouteLocator customRouteLocator(RouteLocatorBuilder builder) {
        return builder.routes()
            .route("payment-service", r -> r
                .path("/api/payments/**")
                .filters(f -> f
                    .circuitBreaker(config -> config
                        .setName("payment-service")
                        .setFallbackUri("forward:/fallback/payment"))
                    .retry(3))
                .uri("lb://payment-service"))
            .route("inventory-service", r -> r
                .path("/api/inventory/**")
                .filters(f -> f
                    .circuitBreaker(config -> config
                        .setName("inventory-service")
                        .setFallbackUri("forward:/fallback/inventory"))
                    .requestRateLimiter())
                .uri("lb://inventory-service"))
            .route("payment-fallback", r -> r
                .path("/fallback/payment")
                .filters(f -> f
                    .setResponseStatus(HttpStatus.SERVICE_UNAVAILABLE)
                    .setResponseBody("{\"error\":\"Payment service unavailable\"}"))
                .uri("no://op"))
            .route("inventory-fallback", r -> r
                .path("/fallback/inventory")
                .filters(f -> f
                    .setResponseStatus(HttpStatus.OK)
                    .setResponseBody("{\"products\":[],\"source\":\"cache\"}"))
                .uri("no://op"))
            .build();
    }
}

Expected HTTP responses:

GET /api/payments/123 -> 200 (normal) or 503 (circuit open)
GET /api/inventory -> 200 with products or 200 with empty array (fallback)

Kong Gateway Circuit Breaker

-- Kong plugin: circuit-breaker per upstream
local circuit_breaker = {
    name = "circuit-breaker",
    config = {
        upstreams = {
            {
                name = "payment-service",
                thresholds = {
                    failures = 5,
                    successes = 3,
                    timeout = 30,
                },
                fallback = {
                    status_code = 503,
                    body = '{"error":"Payment service unavailable"}',
                }
            },
            {
                name = "inventory-service",
                thresholds = {
                    failures = 10,
                    successes = 5,
                    timeout = 60,
                },
                fallback = {
                    status_code = 200,
                    body = '{"data":[],"source":"cache"}',
                }
            }
        },
        half_open_probes = {
            enabled = true,
            interval = 5,
            method = "HEAD",
            path = "/health",
        }
    }
}

print("Circuit breaker configured for payment and inventory upstreams")

Expected behavior:

Request to payment-service after 5 failures: 503 response
Health probe every 5 seconds in half-open state
After 3 successful probes: circuit closes, traffic resumes

Circuit Breaker State Propagation

import time
import requests

class GatewayCircuitBreaker:
    def __init__(self, service_name, fail_threshold=5, recovery_timeout=30):
        self.service_name = service_name
        self.fail_threshold = fail_threshold
        self.recovery_timeout = recovery_timeout
        self.failures = 0
        self.state = 'CLOSED'
        self.last_failure = 0
        self.state_versions = 0

    def should_forward(self):
        if self.state == 'OPEN':
            if time.time() - self.last_failure > self.recovery_timeout:
                self.state = 'HALF_OPEN'
                return True
            return False
        return True

    def record_result(self, success):
        if success:
            self.failures = 0
            if self.state in ('HALF_OPEN', 'OPEN'):
                old_state = self.state
                self.state = 'CLOSED'
                self.state_versions += 1
                print(f"[Gateway] {self.service_name}: {old_state} -> CLOSED")
        else:
            self.failures += 1
            self.last_failure = time.time()
            if self.failures >= self.fail_threshold:
                old_state = self.state
                self.state = 'OPEN'
                self.state_versions += 1
                print(f"[Gateway] {self.service_name}: {old_state} -> OPEN")

    def get_state_info(self):
        return {
            'service': self.service_name,
            'state': self.state,
            'failures': self.failures,
            'version': self.state_versions,
        }

gateway_cb = GatewayCircuitBreaker("payment-service")

for i in range(8):
    forward = gateway_cb.should_forward()
    if forward:
        success = (i < 2)
        gateway_cb.record_result(success)
        print(f"Request {i+1}: forwarded {'(success)' if success else '(failed)'}")
    else:
        print(f"Request {i+1}: blocked (fallback response)")
    time.sleep(0.1)

Expected output:

Request 1: forwarded (success)
Request 2: forwarded (success)
[Gateway] payment-service: CLOSED -> OPEN
Request 3: forwarded (failed)
Request 4: forwarded (failed)
Request 5: forwarded (failed)
Request 6: blocked (fallback response)
Request 7: blocked (fallback response)
Request 8: blocked (fallback response)

Common Mistakes

• Gateway circuit breaker without fallback -- gateway-level errors without fallbacks return 502 Bad Gateway. Always define fallback responses at the gateway level: cached data, error JSON, or redirect to a degraded experience.
• Single circuit breaker for all routes -- one circuit breaker for all services opens for all when one fails. Use per-service circuit breakers so the inventory service fails independently of payments.
• No half-open probe Strategy -- gateways must actively probe recovering services. Configure health check probes that verify actual service health, not just TCP connectivity. Use HEAD requests to lightweight health endpoints.
• Circuit breaker timeout shorter than upstream timeout -- if the upstream service has a 30-second timeout but the circuit breaker waits only 5 seconds before counting failure, slow requests cause false positives. Align timeouts.
• Not propagating circuit breaker state to monitoring -- gateway circuit breakers must expose state through health endpoints, Prometheus metrics, and distributed tracing. Each circuit breaker's state should be visible in your operations dashboard.

Practice Questions

1. Why should gateway circuit breakers be per-service rather than global?
2. What type of fallback responses should a gateway return?
3. How does half-open probing work at the gateway level?
4. How do you propagate circuit breaker state from the gateway to monitoring?
5. How do you coordinate gateway circuit breakers with downstream circuit breakers?

Challenge

Build a gateway resilience configuration for 5 microservices: (1) per-service circuit breakers with different thresholds (payment=3, inventory=5, notification=8, analytics=15, recommendations=10), (2) per-service fallback responses: payment=503 with error, inventory=200 with cached data, notification=202 (queued), analytics=204 (no content), recommendations=200 with popular items, (3) health probe endpoints per service with timeout and interval, (4) circuit breaker metrics exposed via Prometheus endpoint on the gateway, (5) state propagation: gateway circuit breaker state piggybacks on request headers to inform downstream services, (6) rate limiting before circuit breaker to prevent brute force from keeping the circuit open.

FAQ

Mini Project

Build a gateway with full resilience: (1) API gateway with per-service circuit breakers for 4 upstream microservices, (2) each circuit breaker has different thresholds based on service criticality, (3) fallback responses: read services return cached data, write services return 503 with retry-after headers, (4) health probes that call upstream /health endpoints in half-open state, (5) retry (2 attempts) composed before the circuit breaker, (6) rate limiter (100 req/s per service) before the circuit breaker, (7) Prometheus metrics: circuit state per service, request count, blocked count, fallback count, (8) health endpoint on the gateway showing all circuit breaker states.

What's Next

Continue with gRPC Integration to learn gRPC circuit breaker patterns. Then explore Message Queue for Message Broker circuit breaker patterns.