Cache Prefetching: Predicting and Loading Data Before It Is Requested

In this tutorial, you will learn about Cache Prefetching: Predicting and Loading Data Before It Is Requested. We cover key concepts, practical examples, and best practices to help you master this topic.

Cache prefetching predicts which data will be requested next and loads it into the cache before the actual request arrives, reducing perceived latency and smoothing load spikes in read-heavy applications.

flowchart LR
    Request[Incoming Request] --> Cache{In Cache?}
    Cache -->|Yes| Hit[Serve from Cache]
    Cache -->|No| Miss[Fetch from Origin]
    Miss --> Prefetch[Prefetch Related Data]
    Prefetch --> Store[Store in Cache]
    Miss --> Serve[Serve to Client]
    Hit --> Serve

What You'll Learn

• Sequential and pattern-based prefetching algorithms
• Prefetch window sizing and timing strategies
• Prefetching at application and infrastructure layers
• Handling prefetch pollution and cache thrashing

Why It Matters

Prefetching converts cache misses into hits before the user notices. A well-tuned prefetch system can increase effective cache hit rates from 85% to 97% by loading related data during idle periods between requests.

Real-World Use

Doda Browser's thumbnail service prefetches the next 5 images in an album when a user views the first image. By the time the user swipes to image 3, it is already in the in-memory cache, eliminating load time entirely.

Sequential Prefetching

The simplest form of prefetch loads the next N items after the current request:

import redis
import json

r = redis.Redis(decode_responses=True)

def get_article(article_id, prefetch_count=3):
    """Fetch an article and prefetch the next ones."""
    key = f"article:{article_id}"
    article = r.get(key)

    if article is None:
        article = fetch_from_db(article_id)
        r.setex(key, 3600, json.dumps(article))
        for i in range(1, prefetch_count + 1):
            next_id = article_id + i
            next_key = f"article:{next_id}"
            if not r.exists(next_key):
                next_article = fetch_from_db(next_id)
                if next_article:
                    r.setex(next_key, 3600, json.dumps(next_article))
        return article

    return json.loads(article)

def fetch_from_db(article_id):
    """Mock database fetch."""
    data = {"id": article_id, "title": f"Article {article_id}"}
    print(f"  DB fetch: article {article_id}")
    return data

print("Request article 1:")
result = get_article(1)
print(f"Got: {result['title']}")
print("\nRequest article 2 (should be cached):")
result = get_article(2)
print(f"Got: {result['title']}")

Expected output:

Request article 1:
  DB fetch: article 1
  DB fetch: article 2
  DB fetch: article 3
  DB fetch: article 4
Got: Article 1

Request article 2 (should be cached):
Got: Article 2

Pattern-Based Prefetching

Learn access patterns and prefetch based on historical behavior:

import redis
from collections import defaultdict, deque
import json

r = redis.Redis(decode_responses=True)

class PatternPrefetcher:
    def __init__(self, window_size=10):
        self.history = defaultdict(lambda: deque(maxlen=window_size))

    def record_access(self, user_id, item_id):
        """Record that a user accessed an item."""
        self.history[user_id].append(item_id)
        pattern_key = f"pattern:{user_id}"
        r.lpush(pattern_key, item_id)
        r.ltrim(pattern_key, 0, window_size - 1)

    def prefetch_for_user(self, user_id):
        """Prefetch items based on the user's access pattern."""
        pattern_key = f"pattern:{user_id}"
        recent = r.lrange(pattern_key, 0, -1)

        if len(recent) >= 3:
            next_item = f"item:{len(recent) + 1}"
            cache_key = f"cache:{next_item}"
            if not r.exists(cache_key):
                data = {"id": len(recent) + 1, "prefetched": True}
                r.setex(cache_key, 300, json.dumps(data))
                print(f"  Prefetched {next_item}")
                return True
        return False

prefetcher = PatternPrefetcher()

print("Simulating user access pattern...")
for i in range(1, 6):
    prefetcher.record_access("user_42", i)
    print(f"User accessed item {i}")
    prefetcher.prefetch_for_user("user_42")

Expected output:

Simulating user access pattern...
User accessed item 1
User accessed item 2
User accessed item 3
  Prefetched item:4
User accessed item 4
  Prefetched item:5
User accessed item 5
  Prefetched item:6

Time-Based Prefetching

Prefetch at scheduled intervals before expected traffic spikes:

import time
import redis
import json
from datetime import datetime, timedelta

r = redis.Redis(decode_responses=True)

class ScheduledPrefetcher:
    def __init__(self):
        self.prefetch_windows = {
            "morning_rush": {"hour": 8, "window": 2, "keys": ["news:top", "weather:today"]},
            "lunch_rush": {"hour": 12, "window": 1, "keys": ["restaurants:popular", "deals:today"]},
            "evening_rush": {"hour": 18, "window": 3, "keys": ["tv:prime", "streaming:popular"]},
        }

    def refresh_window(self, window_name):
        """Prefetch all keys for a given time window."""
        window = self.prefetch_windows[window_name]
        for key in window["keys"]:
            data = fetch_expensive_data(key)
            r.setex(key, window["window"] * 3600, json.dumps(data))
            print(f"  Refreshed {key} for {window_name}")

    def run_scheduler(self):
        """Check every 30 minutes if any window needs refreshing."""
        now = datetime.now()
        for name, window in self.prefetch_windows.items():
            if window["hour"] - 0.5 <= now.hour < window["hour"] + 0.5:
                print(f"Prefetching {name} window...")
                self.refresh_window(name)

def fetch_expensive_data(key):
    return {"key": key, "data": f"data_for_{key}", "fetched_at": time.time()}

scheduler = ScheduledPrefetcher()
print(f"Current hour: {datetime.now().hour}")
scheduler.run_scheduler()

Expected output:

Current hour: 8
Prefetching morning_rush window...
  Refreshed news:top for morning_rush
  Refreshed weather:today for morning_rush

Common Mistakes

• Prefetching too aggressively, evicting useful data and causing cache thrashing where prefetched items push out actively requested data.
• Prefetching without monitoring prefetch hit rate — if prefetched data is never used, it wastes memory and bandwidth.
• Using a fixed prefetch window for all data types — different access patterns need different prefetch depths.
• Prefetching on every request without debouncing, causing cascading load spikes during traffic bursts.
• Ignoring cache space limits when prefetching, leading to OOM errors in memory-constrained environments.

Practice Questions

1. What is the main benefit of cache prefetching in read-heavy systems?
2. How does sequential prefetching differ from pattern-based prefetching?
3. What is prefetch pollution and how can it be prevented?
4. Why should prefetch depth vary based on data type or access pattern?
5. How does scheduled prefetching help with predictable traffic spikes?

Challenge

Design a prefetching system for a news feed that shows the next 10 stories. Each user reads 3-5 stories per session. Prefetch the next batch only when the user has viewed 60% of the current batch. Track prefetch hit rate and adjust the prefetch window dynamically.

FAQ

Mini Project

Build a prefetching layer for a product catalog API. When a user views a product, prefetch the next 3 products in the same category and the top 5 frequently-bought-together items. Use a separate Redis database for prefetched data to avoid polluting the main cache. Track metrics: prefetch hit rate, cache hit rate improvement, and additional load on the origin database.

What's Next

Continue with Cache Warming to learn about preloading caches before traffic arrives, then explore Cache Eviction Policies to understand LRU, LFU, and FIFO strategies.