Circuit Breaker Pattern in API Gateway — Preventing Cascading Failures

In this tutorial, you will learn about Circuit Breaker Pattern in API Gateway. We cover key concepts, practical examples, and best practices to help you master this topic.

The circuit breaker pattern prevents a failing backend service from causing cascading failures by monitoring error rates, opening the circuit when failures exceed a threshold, and allowing the service time to recover.

What You'll Learn

• The three circuit breaker states: closed, open, half-open
• How to implement a circuit breaker in a gateway
• Combining circuit breakers with fallback responses

Why It Matters

When a backend service starts failing slowly (timeouts, 500 errors), clients retry, making the problem worse. Retries pile up, exhausting connection pools and CPU. The circuit breaker detects the failure pattern, stops sending traffic to the failing service immediately, and lets it recover.

Real-World Use

Durga Antivirus Pro's threat analysis service occasionally becomes overloaded during large malware outbreaks. The gateway's circuit breaker detects 50%+ error rates, opens the circuit for 30 seconds, and serves a cached "scan in progress" response instead of failing outright.

flowchart LR
    subgraph "Circuit Breaker States"
        CL["Closed\nNormal operation"] -->|"Failures > threshold"| OP["Open\nRejecting requests"]
        OP -->|"Timeout elapsed"| HO["Half-Open\nTesting service"]
        HO -->|"Success"| CL
        HO -->|"Failure"| OP
    end

Circuit Breaker Implementation

import time
import threading

class CircuitBreaker:
    def __init__(self, failure_threshold=5, recovery_timeout=30):
        self.failure_threshold = failure_threshold
        self.recovery_timeout = recovery_timeout
        self.failure_count = 0
        self.state = "closed"
        self.last_failure_time = 0
        self.lock = threading.Lock()

    def call(self, func, fallback=None):
        with self.lock:
            if self.state == "open":
                if time.time() - self.last_failure_time > self.recovery_timeout:
                    self.state = "half-open"
                else:
                    return fallback() if fallback else None

        try:
            result = func()
            with self.lock:
                if self.state == "half-open":
                    self.state = "closed"
                    self.failure_count = 0
                return result
        except Exception as e:
            with self.lock:
                self.failure_count += 1
                self.last_failure_time = time.time()
                if self.failure_count >= self.failure_threshold:
                    self.state = "open"
                return fallback() if fallback else None

Using the Circuit Breaker in the Gateway

from flask import Flask, jsonify
import requests

app = Flask(__name__)

def call_threat_service():
    resp = requests.get("http://threat-service:8080/analyze", timeout=5)
    resp.raise_for_status()
    return resp.json()

def fallback_response():
    return {"status": "queued", "message": "Analysis in progress"}

threat_cb = CircuitBreaker(failure_threshold=3, recovery_timeout=30)

@app.route("/api/threat/analyze")
def analyze_threat():
    result = threat_cb.call(call_threat_service, fallback_response)
    return jsonify(result)

Monitoring Breaker State

@app.route("/admin/circuit-breakers")
def breaker_status():
    return jsonify({
        "threat_service": {
            "state": threat_cb.state,
            "failures": threat_cb.failure_count,
        }
    })

Common Mistakes

1. Circuit Breaker Without Fallback

Opening the circuit without a fallback means returning errors. Provide cached responses, default data, or a degradation message.

2. Threshold Too Low or Too High

A threshold of 1 failure opens the circuit during transient issues. A threshold of 100 lets a dying service harm the system. Choose based on normal error rates.

3. Not Logging State Changes

Without logging, you cannot detect when circuits open. Log every state transition for operational visibility.

4. Shared Circuit State Across Instances

Each gateway instance has its own circuit state. A failing service may be called by other instances that haven't detected the failure yet. Use distributed circuit state via Redis.

5. No Manual Override

Operations teams need to manually open or close a circuit during incidents. Provide an admin API for manual control.

Practice Questions

1. What are the three states of a circuit breaker?
2. Why does the circuit breaker prevent cascading failures?
3. What happens when the circuit is half-open?
4. Why is a fallback important when the circuit is open?
5. How can you share circuit breaker state across multiple gateway instances?

Answers:

1. Closed (normal operation), Open (rejecting requests), Half-Open (testing if service recovered).
2. By stopping traffic to a failing service, the circuit breaker prevents resource exhaustion (connection pools, threads, CPU) from affecting other services.
3. The circuit allows a single request through. If it succeeds, the circuit closes. If it fails, the circuit reopens.
4. A fallback returns a degraded but useful response instead of an error, maintaining partial functionality for users.
5. Store failure counts and state in Redis or another distributed store that all gateway instances can read and write.

Challenge: Implement a distributed circuit breaker using Redis. Store failure counts with TTL so that failures older than 60 seconds are automatically reset.

FAQ

Mini Project

Build a Flask gateway with circuit breakers for three backend services. Each breaker has a different threshold and recovery timeout. Add an admin endpoint to view breaker states and a manual API to open/close circuits.

What's Next

Continue with Caching in API Gateway to improve response times, or explore IP Whitelisting in Gateway for access control.