Cache Persistence: RDB Snapshots and AOF Logs for Redis Durability
In this tutorial, you will learn about Cache Persistence: RDB Snapshots and AOF Logs for Redis Durability. We cover key concepts, practical examples, and best practices to help you master this topic.
Redis persistence ensures cached data survives restarts through RDB snapshots (point-in-time backups) and AOF logs (append-only command journals), letting you balance recovery speed against data durability for your cache workloads.
flowchart TD
Redis[Redis Process] --> RDB[RDB Snapshot]
Redis --> AOF[AOF Log]
RDB -->|fork + dump| Disk1[dump.rdb]
AOF -->|append commands| Disk2[appendonly.aof]
Disk1 -->|load on restart| Recovery[Recovery]
Disk2 -->|replay on restart| Recovery
Recovery --> Ready[Redis Ready with Cached Data]
What You'll Learn
- RDB snapshots: configuration, frequency, and performance impact
- AOF logs: fsync policies, rewrite, and durability guarantees
- Choosing between RDB, AOF, or both for cache workloads
- Recovery procedures and trade-offs
Why It Matters
Without persistence, a Redis restart after a power failure or crash means an empty cache. Every request becomes a cache miss, triggering a thundering herd against your database. Persistence reduces the warm-up window from hours to minutes by preserving cached data across restarts.
Real-World Use
DodaTech's session cache runs with RDB snapshots every 5 minutes and AOF with fsync every second. After an unexpected power outage, Redis recovers in 45 seconds with 6 seconds of data loss (the gap between the last AOF fsync and the crash). Without persistence, recovery would take 20+ minutes of database hammering.
RDB Snapshot Configuration
Configure periodic snapshots for backup and recovery:
import redis
import time
import json
r = redis.Redis(decode_responses=True)
class RDBSnapshotManager:
def __init__(self, redis_client):
self.r = redis_client
def trigger_save(self):
"""Trigger a synchronous RDB save (blocks Redis)."""
start = time.time()
self.r.save()
elapsed = time.time() - start
return {"method": "SAVE", "elapsed_seconds": round(elapsed, 2)}
def trigger_bgsave(self):
"""Trigger an asynchronous RDB save (forks, non-blocking)."""
result = self.r.bgsave()
if result:
return {"method": "BGSAVE", "status": "started"}
return {"method": "BGSAVE", "status": "already_running"}
def last_save_info(self):
"""Get information about the last successful save."""
info = self.r.info("persistence")
return {
"rdb_last_save_time": info.get("rdb_last_save_time", 0),
"rdb_last_bgsave_status": info.get("rdb_last_bgsave_status", "unknown"),
"rdb_last_bgsave_time_sec": info.get("rdb_last_bgsave_time_sec", -1),
"rdb_current_bgsave_time_sec": info.get("rdb_current_bgsave_time_sec", -1),
"rdb_changes_since_last_save": info.get("rdb_changes_since_last_save", 0),
}
def estimate_rdb_size(self):
"""Estimate the RDB file size based on current memory usage."""
info = self.r.info("memory")
memory_used = info.get("used_memory", 0)
rdb_estimate = memory_used * 1.1
return {
"used_memory": memory_used,
"estimated_rdb_bytes": int(rdb_estimate),
"estimated_rdb_mb": round(rdb_estimate / 1024 / 1024, 2),
}
manager = RDBSnapshotManager(r)
result = manager.trigger_bgsave()
print(f"BGSAVE: {result['status']}")
save_info = manager.last_save_info()
print(f"Last save: {save_info['rdb_last_save_time']}")
print(f"Last BGSAVE time: {save_info['rdb_last_bgsave_time_sec']}s")
print(f"Changes since last save: {save_info['rdb_changes_since_last_save']}")
rdb_est = manager.estimate_rdb_size()
print(f"Estimated RDB size: {rdb_est['estimated_rdb_mb']} MB")
Expected output:
BGSAVE: started
Last save: 1719580800
Last BGSAVE time: 2s
Changes since last save: 0
Estimated RDB size: 45.32 MB
AOF Configuration and Monitoring
Configure and monitor the Append Only File:
import redis
r = redis.Redis(decode_responses=True)
class AOFManager:
def __init__(self, redis_client):
self.r = redis_client
def get_aof_info(self):
"""Get AOF status and configuration."""
info = self.r.info("persistence")
return {
"aof_enabled": info.get("aof_enabled", 0) == 1,
"aof_fsync": self.get_config("appendfsync"),
"aof_current_size": info.get("aof_current_size", 0),
"aof_base_size": info.get("aof_base_size", 0),
"aof_delayed_fsync": info.get("aof_delayed_fsync", 0),
}
def get_config(self, param):
"""Get a Redis configuration parameter."""
try:
return self.r.config_get(param).get(param, "unknown")
except:
return "unknown"
def recommend_fsync_policy(self, workload_type):
"""Recommend an fsync policy based on workload."""
policies = {
"cache_high_throughput": {
"policy": "everysec",
"reason": "Best balance: at most 1 second of data loss"
},
"cache_max_durability": {
"policy": "always",
"reason": "Every write is fsynced, but 100-1000x slower writes"
},
"cache_performance": {
"policy": "no",
"reason": "OS controls flush, fastest but up to 30s data loss"
},
}
return policies.get(workload_type, policies["cache_high_throughput"])
def estimate_rewrite_benefit(self):
"""Estimate how much AOF rewrite would reduce file size."""
info = self.r.info("persistence")
current = info.get("aof_current_size", 0)
base = info.get("aof_base_size", 0)
if base > 0 and current > base:
reduction = (current - base) / current * 100
return {
"current_size_mb": round(current / 1024 / 1024, 2),
"base_size_mb": round(base / 1024 / 1024, 2),
"potential_reduction_percent": round(reduction, 1),
}
return {"message": "AOF rewrite not needed (current size at base)"}
aof = AOFManager(r)
info = aof.get_aof_info()
print(f"AOF enabled: {info['aof_enabled']}")
print(f"fsync policy: {info['aof_fsync']}")
print(f"AOF current size: {info['aof_current_size']} bytes")
rec = aof.recommend_fsync_policy("cache_high_throughput")
print(f"Recommended policy: {rec['policy']} ({rec['reason']})")
benefit = aof.estimate_rewrite_benefit()
if "potential_reduction_percent" in benefit:
print(f"AOF rewrite could reduce {benefit['potential_reduction_percent']}%")
Expected output:
AOF enabled: True
fsync policy: everysec
AOF current size: 2500000 bytes
Recommended policy: everysec (Best balance: at most 1 second of data loss)
AOF rewrite could reduce 35.2%
Recovery Simulation
Simulate saving and reloading cache data:
import redis
import json
import time
r = redis.Redis(decode_responses=True)
class CacheRecoverySimulator:
def __init__(self, redis_client):
self.r = redis_client
def snapshot_and_verify(self, test_keys):
"""Save data and verify it persists."""
print("Step 1: Saving data...")
for key, value in test_keys.items():
self.r.setex(key, 3600, json.dumps(value))
print(f" Stored {len(test_keys)} keys")
pre_save_count = self.r.dbsize()
print(f" DB size: {pre_save_count}")
print("\nStep 2: Triggering RDB save...")
self.r.bgsave()
time.sleep(1)
save_info = self.r.info("persistence")
print(f" Last save status: {save_info['rdb_last_bgsave_status']}")
print("\nStep 3: Verifying data is in RDB...")
saved_keys = list(test_keys.keys())
loaded = {k: json.loads(self.r.get(k)) for k in saved_keys if self.r.get(k)}
print(f" Recovered {len(loaded)} of {len(saved_keys)} keys")
return {
"saved": len(test_keys),
"recovered": len(loaded),
"success": len(loaded) == len(test_keys),
"db_size": pre_save_count,
}
simulator = CacheRecoverySimulator(r)
test_data = {
"recovery:user:1": {"name": "Alice", "email": "alice@example.com"},
"recovery:user:2": {"name": "Bob", "email": "bob@example.com"},
"recovery:config:theme": "dark",
"recovery:session:abc": {"token": "xyz", "expires": 3600},
}
result = simulator.snapshot_and_verify(test_data)
print(f"\nRecovery test {'PASSED' if result['success'] else 'FAILED'}")
print(f"Keys recovered: {result['recovered']}/{result['saved']}")
Expected output:
Step 1: Saving data...
Stored 4 keys
DB size: 150
Step 2: Triggering RDB save...
Last save status: ok
Step 3: Verifying data is in RDB...
Recovered 4 of 4 keys
Recovery test PASSED
Keys recovered: 4/4
Common Mistakes
- Disabling persistence entirely in production — a single crash causes complete cache loss. Always enable at least RDB snapshots for recovery.
- Using AOF with fsync always in high-throughput caches — fsync on every write reduces throughput by 100-1000x. Use everysec for cache workloads.
- Setting RDB save intervals too frequently (every 30 seconds) — forking a large Redis Process every 30 seconds causes latency spikes from copy-on-write overhead.
- Not monitoring AOF rewrite progress — a stalled AOF rewrite can grow the AOF file unboundedly, eventually filling the disk.
- Running Redis with both persistence disabled and maxmemory-policy noeviction — this guarantees data loss on restart and writes failing when memory fills up.
Practice Questions
- What is the difference between RDB and AOF persistence?
- Why does AOF with fsync always reduce write throughput significantly?
- When would you use both RDB and AOF together?
- What happens to Redis performance during a BGSAVE fork?
- How does AOF rewrite reduce the log file size?
Challenge
Design a persistence Strategy for a cache that stores 50 GB of data across 6 Redis Cluster nodes. The SLA requires: maximum 5 seconds of data loss on crash, recovery within 2 minutes, and no more than 10% throughput reduction during persistence. Choose RDB interval, AOF fsync policy, and rewrite configuration. Justify each choice with performance numbers.
FAQ
Mini Project
Build a persistence benchmark tool that: (1) populates Redis with 1 million keys of varying sizes, (2) measures throughput with persistence disabled (baseline), (3) enables RDB with 5-minute save intervals and measures throughput impact, (4) enables AOF with everysec fsync and measures throughput, (5) enables both and measures throughput, and (6) reports the overhead percentage for each persistence configuration.
What's Next
Continue with Cache Transactions to learn about atomic cache operations with MULTI/EXEC. Then explore Cache Locking for distributed locks with Redis.
Built by the developers of DodaTech
Doda Browser, DodaZIP & Durga Antivirus Pro