Cache Stampede

title: "Cache Stampede — Preventing Thundering Herd in API Caching" description: "Cache stampede occurs when a popular cached key expires and multiple simultaneous requests all hit the origin server, causing a thundering herd of database load." date: 2026-06-28 lastmod: 2026-06-28 weight: 22 tags: [apis, caching] }

Cache stampede (thundering herd) happens when concurrent requests for an expired cache key all miss simultaneously and hit the origin server, overwhelming it.

What You'll Learn

• The cache stampede problem
• Early recomputation and locking
• Probabilistic expiration

Why It Matters

A cache stampede on a popular key can multiply your database load by 100x, causing cascading failures and downtime.

Stampede Visualization

sequenceDiagram
    participant R1 as Request 1
    participant R2 as Request 2
    participant R3 as Request 3
    participant C as Cache
    participant DB as Database

    R1->>C: GET key
    R2->>C: GET key
    R3->>C: GET key
    Note over C: Key expired
    C-->>R1: MISS
    C-->>R2: MISS
    C-->>R3: MISS
    R1->>DB: QUERY
    R2->>DB: QUERY
    R3->>DB: QUERY
    Note over DB: OVERLOADED!

Code Examples

import threading
import random

# Cache stampede solution: locking
lock = threading.Lock()

@app.route('/api/popular')
def get_popular():
    cached = cache.get('popular')
    if cached:
        return jsonify(json.loads(cached))

    # Only one request fetches from DB
    if lock.acquire(blocking=False):
        try:
            data = db.execute("SELECT * FROM products ORDER BY views DESC LIMIT 100")
            cache.setex('popular', 300, json.dumps(data))
            return jsonify(data)
        finally:
            lock.release()
    else:
        # Other requests wait briefly or retry
        time.sleep(0.1)
        cached = cache.get('popular')
        if cached:
            return jsonify(json.loads(cached))
        return jsonify(db.execute("SELECT * FROM products ORDER BY views DESC LIMIT 100"))

# Probabilistic early recomputation
def get_or_compute(key, fetch_func, ttl=300):
    cached = cache.get(key)
    if cached:
        data = json.loads(cached)
        # Randomly recompute before expiry
        age = cache.ttl(key)
        if age < ttl * 0.1:  # 10% of TTL remaining
            if random.random() < 0.1:  # 10% of requests recompute
                data = fetch_func()
                cache.setex(key, ttl, json.dumps(data))
        return data
    data = fetch_func()
    cache.setex(key, ttl, json.dumps(data))
    return data

# Stale-while-revalidate
def get_with_stale(key, fetch_func, ttl=300, stale_ttl=600):
    cached = cache.get(key)
    if cached:
        data = json.loads(cached)
        age = cache.ttl(key)
        if age < 0:  # Expired but within stale window
            # Serve stale and refresh in background
            thread = threading.Thread(target=lambda: cache.setex(
                key, ttl, json.dumps(fetch_func())
            ))
            thread.start()
            return data
        return data
    data = fetch_func()
    cache.setex(key, ttl, json.dumps(data))
    return data

Common Mistakes

1. No Locking on Recompute

Multiple requests all hit the database simultaneously.

2. Lock Without Timeout

A slow recompute blocks all subsequent requests indefinitely.

3. Global Lock Instead of Per-Key

One key recompute blocks all other cache operations.

4. No Fallback for Lock Failure

Requests should serve stale data or wait briefly.

5. Long TTL with No Recompute Strategy

Popular items face stampede every TTL cycle.

Practice Questions

1. What causes a cache stampede?
2. How does locking prevent cache stampede?
3. What is probabilistic early recomputation?
4. What is stale-while-revalidate?
5. How does TTL randomization help?

Answers:

1. Multiple concurrent requests for an expired key all miss at the same time.
2. Only one request recomputes the value; others wait or use stale data.
3. Randomly recompute before expiry to spread the load.
4. Serve stale data while asynchronously refreshing the cache.
5. Add random jitter to TTLs so keys expire evenly over time.

Challenge: Implement a cache stampede prevention system for a highly popular API endpoint. Compare behavior with and without locking.

FAQ