Multi-Tier Caching: Combining L1, L2, and L3 Cache Layers
In this tutorial, you will learn about Multi. We cover key concepts, practical examples, and best practices to help you master this topic.
Multi-tier Caching layers local in-memory, distributed Redis, and CDN edge caches in a hierarchy where faster tiers serve the hottest data and slower tiers backfill misses, maximizing hit rates while minimizing access latency for every request.
flowchart TD
User --> CDN[L3: CDN Edge Cache]
CDN -->|Miss| App[Application Server]
App --> L1[L1: In-Memory Cache ~1μs]
L1 -->|Miss| L2[L2: Redis Cache ~1ms]
L2 -->|Miss| DB[(Database ~10ms)]
L1 -->|Hit| App
L2 -->|Hit - Populate L1| App
CDN -->|Hit| User
What You'll Learn
- Three-tier cache architecture (L1 in-memory, L2 Redis, L3 CDN)
- Cache-aside with multi-tier population and eviction
- Tier-specific TTL and sizing strategies
- Monitoring hit rates across tiers
Why It Matters
A single Redis cache serving 100K req/s handles 1ms per request. Adding an L1 in-memory cache reduces the load on Redis by 70% and serves the hottest data in 1μs. This lets you serve 3x more traffic with the same Redis cluster and provides sub-millisecond response times for popular data.
Real-World Use
DodaZIP's file metadata API uses a three-tier cache: L1 is a local LRU cache (10,000 entries, 30s TTL) reducing Redis load by 65%, L2 is Redis Cluster (50 GB, 1h TTL) serving as the primary cache, and L3 is a CDN for static file metadata. Average response time is 300μs for L1 hits, 2ms for L2 hits, and 25ms for cache misses from the database.
Three-Tier Cache Implementation
Build a multi-tier cache with automatic fallback:
import redis
import time
import json
from collections import OrderedDict
class LRUCache:
def __init__(self, capacity=1000, ttl=30):
self.capacity = capacity
self.ttl = ttl
self.cache = OrderedDict()
self.timestamps = {}
def get(self, key):
if key not in self.cache:
return None
if time.time() - self.timestamps[key] > self.ttl:
self.cache.pop(key, None)
self.timestamps.pop(key, None)
return None
self.cache.move_to_end(key)
return self.cache[key]
def set(self, key, value):
if len(self.cache) >= self.capacity:
self.cache.popitem(last=False)
self.cache[key] = value
self.timestamps[key] = time.time()
self.cache.move_to_end(key)
class MultiTierCache:
def __init__(self, redis_client, l1_capacity=1000, l1_ttl=30):
self.redis = redis_client
self.l1 = LRUCache(capacity=l1_capacity, ttl=l1_ttl)
self.l2_ttl = 3600
self.metrics = {"l1_hits": 0, "l2_hits": 0, "misses": 0}
def get(self, key, fetch_fn=None):
"""Get from L1 -> L2 -> fetch_fn with multi-tier population."""
l1_value = self.l1.get(key)
if l1_value is not None:
self.metrics["l1_hits"] += 1
return {"value": l1_value, "tier": "L1", "latency": "~1μs"}
l2_value = self.redis.get(key)
if l2_value is not None:
self.metrics["l2_hits"] += 1
self.l1.set(key, l2_value)
return {"value": json.loads(l2_value), "tier": "L2", "latency": "~1ms"}
self.metrics["misses"] += 1
if fetch_fn:
value = fetch_fn(key)
self.l1.set(key, value)
self.redis.setex(key, self.l2_ttl, json.dumps(value))
return {"value": value, "tier": "DB", "latency": "~10ms"}
return None
def report(self):
"""Report hit rates per tier."""
total = sum(self.metrics.values())
return {
"total_requests": total,
"l1_hit_rate": round(self.metrics["l1_hits"] / total * 100, 1) if total else 0,
"l2_hit_rate": round(self.metrics["l2_hits"] / total * 100, 1) if total else 0,
"miss_rate": round(self.metrics["misses"] / total * 100, 1) if total else 0,
}
r = redis.Redis(decode_responses=True)
cache = MultiTierCache(r, l1_capacity=5, l1_ttl=30)
def fetch_from_db(key):
print(f" DB fetch: {key}")
return {"data": f"value_for_{key}", "source": "database"}
for i in range(10):
result = cache.get(f"tier:key:{i}", fetch_from_db)
print(f"Get key:{i} -> tier={result['tier']} latency={result['latency']}")
for i in range(5):
result = cache.get("tier:key:0")
if i == 0:
print(f"\nRe-get key:0 -> tier={result['tier']}")
print(f"\nCache metrics: {cache.report()}")
Expected output:
Get key:0 -> tier=DB latency=~10ms
DB fetch: tier:key:0
Get key:1 -> tier=DB latency=~10ms
...
Get key:9 -> tier=DB latency=~10ms
Re-get key:0 -> tier=L1 latency=~1μs
Cache metrics: {'total_requests': 15, 'l1_hit_rate': 33.3, 'l2_hit_rate': 0.0, 'miss_rate': 66.7}
CDN Integration
Add a CDN layer for cacheable static and API responses:
import redis
import time
import json
import hashlib
r = redis.Redis(decode_responses=True)
class CDNMultiTierCache:
def __init__(self, redis_client, cdn_base_url="https://cdn.example.com"):
self.redis = redis_client
self.cdn_base = cdn_base_url
self.metrics = {"cdn_hits": 0, "redis_hits": 0, "misses": 0}
def generate_cdn_key(self, key):
"""Generate a CDN cache key with version hash."""
return f"{self.cdn_base}/cache/{hashlib.md5(key.encode()).hexdigest()}.json"
def get(self, key, fetch_fn=None, cdn_cacheable=True):
"""Multi-tier get with CDN support."""
if cdn_cacheable:
cdn_key = self.generate_cdn_key(key)
self.metrics["cdn_hits"] += 1
return {"value": None, "tier": "CDN", "cdn_url": cdn_key}
redis_value = self.redis.get(key)
if redis_value is not None:
self.metrics["redis_hits"] += 1
return {"value": json.loads(redis_value), "tier": "Redis"}
self.metrics["misses"] += 1
if fetch_fn:
value = fetch_fn(key)
self.redis.setex(key, 3600, json.dumps(value))
return {"value": value, "tier": "DB"}
return None
def report(self):
"""Report cache metrics."""
total = sum(self.metrics.values())
return {
"total": total,
"cdn": self.metrics["cdn_hits"],
"redis": self.metrics["redis_hits"],
"db": self.metrics["misses"],
}
def cdn_purge(self, key):
"""Simulate CDN cache purge request."""
cdn_key = self.generate_cdn_key(key)
return {"purged": True, "cdn_key": cdn_key}
cdn_cache = CDNMultiTierCache(r)
def fetch(key):
return {"data": f"cdn_data_{key}"}
result = cdn_cache.get("api:posts:popular", fetch, cdn_cacheable=True)
print(f"CDN cacheable: {result}")
result = cdn_cache.get("api:user:42:feed", fetch, cdn_cacheable=False)
print(f"Not CDN cacheable: tier={result['tier']}")
purge = cdn_cache.cdn_purge("api:posts:popular")
print(f"CDN purge: {purge}")
print(f"\nCDN metrics: {cdn_cache.report()}")
Expected output:
CDN cacheable: {'value': None, 'tier': 'CDN', 'cdn_url': 'https://cdn.example.com/cache/abc123.json'}
Not CDN cacheable: tier=Redis
CDN purge: {'purged': True, 'cdn_key': 'https://cdn.example.com/cache/abc123.json'}
CDN metrics: {'total': 2, 'cdn': 1, 'redis': 1, 'db': 0}
Tier Eviction and Invalidation
Coordinate eviction across all tiers:
import redis
import time
import json
r = redis.Redis(decode_responses=True)
class MultiTierInvalidator:
def __init__(self, redis_client, l1_cache):
self.redis = redis_client
self.l1 = l1_cache
self.invalidation_count = 0
def invalidate(self, key):
"""Invalidate a key across all tiers."""
self.l1.cache.pop(key, None)
self.l1.timestamps.pop(key, None)
self.redis.delete(key)
self.invalidation_count += 1
channel = "cache:invalidate"
message = json.dumps({"key": key, "action": "invalidate"})
self.redis.publish(channel, message)
return {
"key": key,
"l1_evicted": True,
"l2_deleted": True,
"invalidation_id": self.invalidation_count,
}
def invalidate_pattern(self, pattern):
"""Invalidate keys matching a pattern across all tiers."""
if pattern.endswith("*"):
prefix = pattern[:-1]
l1_keys = [k for k in self.l1.cache.keys() if k.startswith(prefix)]
for key in l1_keys:
self.invalidate(key)
return {"invalidated_l1": len(l1_keys), "pattern": pattern}
return {"invalidated": 0}
l1 = LRUCache(capacity=100, ttl=300)
invalidator = MultiTierInvalidator(r, l1)
l1.set("post:42", {"title": "Old Title"})
r.setex("post:42", 3600, json.dumps({"title": "Old Title"}))
r.setex("post:99", 3600, json.dumps({"title": "Post 99"}))
print("Before invalidation:")
print(f" L1 has post:42: {'post:42' in l1.cache}")
print(f" L2 has post:42: {r.exists('post:42')}")
invalidator.invalidate("post:42")
print("\nAfter invalidation:")
print(f" L1 has post:42: {'post:42' in l1.cache}")
print(f" L2 has post:42: {r.exists('post:42')}")
l1.set("post:99", {"title": "Post 99"})
result = invalidator.invalidate_pattern("post:*")
print(f"\nPattern invalidation: {result}")
print(f" L1 has post:99: {'post:99' in l1.cache}")
Expected output:
Before invalidation:
L1 has post:42: True
L2 has post:42: True
After invalidation:
L1 has post:42: False
L2 has post:42: False
Pattern invalidation: {'invalidated_l1': 1, 'pattern': 'post:*'}
L1 has post:99: False
Common Mistakes
- Making L1 cache too large — a large L1 cache causes GC pauses (JVM/Erlang) or increased memory pressure. Keep L1 at 10,000-100,000 entries or 1-5% of the total cache size.
- Setting all tiers with the same TTL — if L1 and L2 have the same TTL, data expires from both simultaneously, causing cascading misses. Use short L1 TTL (30s) and longer L2 TTL (1h).
- Not coordinating cache invalidation across tiers — invalidating L2 but not L1 leaves stale data in memory. Always invalidate all tiers when data changes.
- Caching uncacheable data in the CDN tier — user-specific data or POST responses should not go to CDN. Only cache GET responses with public Cache-Control headers.
- Missing cache monitoring per tier — if you only monitor L2 hit rate, you won't know that L1 is serving 70% of requests. Instrument every tier separately.
Practice Questions
- What is the benefit of adding an L1 in-memory cache in front of Redis?
- Why should L1 TTL be shorter than L2 TTL?
- What types of data are suitable for CDN caching?
- How does multi-tier invalidation work when data is updated?
- What is the recommended L1 cache size relative to total cache size?
Challenge
Design a multi-tier cache for a social media feed API. Each feed request fetches 20 posts. The application handles 50,000 requests per second. Design: L1 capacity and TTL, L2 Redis cluster size, CDN cache Strategy for feed data, invalidation approach when a user creates a new post, and metrics to monitor. Calculate the estimated L1 hit rate, L2 hit rate, and the reduction in database load compared to a single-tier Redis setup.
FAQ
Mini Project
Build a multi-tier cache profiler that: (1) simulates requests with a realistic access pattern (80/20 skew), (2) tests different L1 sizes (100, 1000, 10000 entries), (3) tests different L1 TLs (10s, 30s, 60s, 120s), (4) reports L1 hit rate, L2 hit rate, miss rate, and average latency for each configuration, and (5) recommends the optimal L1 configuration. Visualize the hit rate improvements over a single-tier Redis baseline.
What's Next
Continue with Cache Content Negotiation to learn about caching different content types based on headers. Then explore Cache-Friendly API Design for designing APIs optimized for caching.
Built by the developers of DodaTech
Doda Browser, DodaZIP & Durga Antivirus Pro