Circuit Breaker Cache Patterns — Optimizing Performance with Cached Fallbacks

In this tutorial, you will learn about Circuit Breaker Cache Patterns. We cover key concepts, practical examples, and best practices to help you master this topic.

Circuit breaker cache integration combines Caching with circuit breaking to serve stale data during outages, reduce load on recovering services, prevent cache stampedes, and maintain performance when downstream services are degraded.

flowchart LR
    R[Request] --> Cache{Cache Hit?}
    Cache -->|Yes, Fresh| Return[Return Cached]
    Cache -->|No| CB{Circuit Open?}
    CB -->|No| Service[Call Service]
    Service -->|Success| Update[Update Cache]
    Service -->|Fail| Fallback[Return Stale Cache]
    CB -->|Yes| Stale[Return Stale Cache]

What You'll Learn

• Cache-aside with circuit breaker fallback
• Read-through cache patterns
• Cache stampede prevention
• Distributed cache consistency
• Cache invalidation on recovery

Why It Matters

Caching without circuit awareness can cause cache stampedes when a service recovers, or serve stale data indefinitely. Integrating cache with circuit breakers ensures caches protect both the application and the downstream service during failures.

Real-World Use

DodaTech's product API uses a cache-aside pattern with a 5-minute TTL. When the catalog service fails, the circuit breaker opens and cached data serves for up to 1 hour with staleness headers. This reduced catalog service load by 60% during recovery.

Cache-Aside with Circuit Breaker

import time
import threading
import random

class CacheEntry:
    def __init__(self, value, ttl):
        self.value = value
        self.expires_at = time.time() + ttl

class CircuitBreakerCache:
    def __init__(self, default_ttl=300, stale_ttl=3600):
        self.cache = {}
        self.default_ttl = default_ttl
        self.stale_ttl = stale_ttl
        self.failures = 0
        self.state = 'CLOSED'
        self.lock = threading.Lock()

    def get(self, key, fetch_fn):
        with self.lock:
            entry = self.cache.get(key)

        if entry and time.time() < entry.expires_at:
            print(f"Cache HIT: {key}")
            return {"value": entry.value, "source": "cache"}

        if entry and time.time() < entry.expires_at + self.stale_ttl:
            if self.state == 'OPEN':
                print(f"Cache STALE (circuit open): {key}")
                return {"value": entry.value, "source": "stale"}
            if random.random() < 0.1:
                print(f"Cache STALE (probabilistic): {key}")
                return {"value": entry.value, "source": "stale"}

        return self._fetch_and_cache(key, fetch_fn)

    def _fetch_and_cache(self, key, fetch_fn):
        try:
            value = fetch_fn()
            with self.lock:
                self.cache[key] = CacheEntry(value, self.default_ttl)
            self.failures = 0
            self.state = 'CLOSED'
            print(f"Cache MISS (fetched): {key}")
            return {"value": value, "source": "live"}
        except Exception as e:
            self.failures += 1
            if self.failures >= 3:
                self.state = 'OPEN'
            entry = self.cache.get(key)
            if entry:
                print(f"Cache MISS (serving stale): {key}")
                return {"value": entry.value, "source": "stale"}
            raise

cache = CircuitBreakerCache(default_ttl=5, stale_ttl=60)

def fetch_product(id):
    if random.random() < 0.5:
        raise Exception("Service unavailable")
    return {"id": id, "name": f"Product {id}", "price": 19.99}

for i in range(8):
    result = cache.get("product:1", lambda: fetch_product(1))
    print(f"Request {i+1}: source={result['source']}")
    time.sleep(0.1)

Expected output:

Cache MISS (fetched): product:1
Request 1: source=live
Cache HIT: product:1
Request 2: source=cache
Cache HIT: product:1
Request 3: source=cache
Cache STALE (circuit open): product:1
Request 4: source=stale
...

Read-Through Cache

import time

class ReadThroughCache:
    def __init__(self, load_fn, ttl=60, max_stale=600):
        self.load_fn = load_fn
        self.ttl = ttl
        self.max_stale = max_stale
        self.cache = {}
        self.failures = 0
        self.state = 'CLOSED'
        self.loading = {}

    def get(self, key):
        now = time.time()
        entry = self.cache.get(key)

        if entry and now < entry['expires']:
            return entry['value']

        if entry and now < entry['expires'] + self.max_stale:
            if self.state == 'OPEN':
                return entry['value']

        if key not in self.loading:
            self.loading[key] = True
            try:
                value = self.load_fn(key)
                self.cache[key] = {'value': value, 'expires': now + self.ttl}
                self.failures = 0
                self.state = 'CLOSED'
                return value
            except Exception:
                self.failures += 1
                if self.failures >= 3:
                    self.state = 'OPEN'
                if entry:
                    return entry['value']
                raise
            finally:
                del self.loading[key]

        time.sleep(0.05)
        return self.get(key)

def load_user(user_id):
    time.sleep(0.1)
    if random.random() < 0.3:
        raise Exception("DB unavailable")
    return {"id": user_id, "name": f"User {user_id}"}

cache = ReadThroughCache(load_user, ttl=5, max_stale=300)

for i in range(6):
    try:
        user = cache.get(42)
        print(f"Request {i+1}: {user['name']}")
    except Exception as e:
        print(f"Request {i+1}: Failed - {e}")
    time.sleep(0.1)

Expected output:

Request 1: User 42
Request 2: User 42
Request 3: User 42
Request 4: User 42  (stale)
Request 5: User 42  (stale, circuit open)
Request 6: User 42  (stale, circuit open)

Common Mistakes

• No stale-while-revalidate in cache -- without stale serving, every cache miss during an outage hits the failing service. Always allow serving stale data up to a max_stale duration when the circuit is open.
• Cache stampede on recovery -- when the circuit closes, many requests simultaneously miss cache and call the service. Use probabilistic early expiration or request coalescing to prevent stampedes.
• Caching write operations -- caching write results can cause stale data serving. Never cache write operations. Cache only read operations and invalidate related cache entries on writes.
• Global cache TTL for all endpoints -- different data has different freshness requirements. Product names can be cached for hours, but stock levels need seconds. Configure per-endpoint TTL.
• Not invalidating cache on circuit recovery -- when the circuit closes after recovery, stale cache entries may persist. Implement proactive cache warming or immediate invalidation on state transitions.

Practice Questions

1. What is the cache-aside pattern and how does it integrate with circuit breakers?
2. How does stale-while-revalidate improve availability during circuit open?
3. What is a cache stampede and how do you prevent it?
4. Why should cache TTL vary by data type?
5. How do you invalidate cache entries when a circuit recovers?

Challenge

Build a cache system with circuit breaker integration: (1) cache-aside with TTL per endpoint and max_stale for degraded serving, (2) probabilistic early expiration that refreshes cache at random intervals before TTL expires (prevents stampedes), (3) circuit breaker integration that extends max_stale when circuit is open (1 hour stale during outage vs 5 minutes normally), (4) write-through invalidation that clears related cache entries on data mutations, (5) cache warming when circuit transitions from open to closed, (6) distributed cache with Redis and local L1 cache (Caffeine-style), (7) metrics: hit rate, stale serve rate, refresh rate, invalidation count.

FAQ

Mini Project

Build a circuit-breaker-aware caching proxy: (1) configurable per-endpoint cache TTL and stale thresholds, (2) circuit breaker per data source that controls cache refresh behavior, (3) two-tier cache: L1 (local in-memory, 10ms) and L2 (Redis, 1ms), (4) probabilistic early expiration (refresh 10% of entries before TTL), (5) stale-while-revalidate with circuit breaker awareness (extend stale window when circuit is open), (6) write-through invalidation that clears L1 and L2 on mutations, (7) recovery warming that proactively refreshes top-100 cache keys when circuit closes, (8) Prometheus metrics for cache hit rate per tier, stale serve rate, refresh rate, and invalidation events.

What's Next

Continue with Resilience4j to learn Java-based circuit breaker implementation. Then explore Hystrix Migration for migrating from Hystrix to Resilience4j.