Cache Locking: Distributed Locks with Redis and Redlock
In this tutorial, you will learn about Cache Locking: Distributed Locks with Redis and Redlock. We cover key concepts, practical examples, and best practices to help you master this topic.
Distributed cache locking coordinates access to shared resources across multiple application instances using Redis-based locks, preventing duplicate work and ensuring data consistency without the overhead of database-level locks.
flowchart TD
App1[App Instance 1] -->|Acquire Lock| Redis[Redis Lock]
App2[App Instance 2] -->|Acquire Lock| Redis
App3[App Instance 3] -->|Acquire Lock| Redis
Redis -->|Lock Acquired| Lock1[Instance 1 Gets Lock]
Redis -->|Lock Denied| Lock2[Instance 2 Retries]
Redis -->|Lock Denied| Lock3[Instance 3 Retries]
Lock1 --> Work[Do Expensive Work]
Work --> Release[Release Lock]
Release --> Lock2
Lock2 --> Work
What You'll Learn
- Basic distributed locking with SET NX EX
- The Redlock algorithm for multi-node safety
- Lock timeout, renewal, and release patterns
- Fencing tokens for strong consistency guarantees
Why It Matters
Without distributed locks, two application instances can simultaneously Process the same job, send the same email, or double-charge a customer. Redis locks provide a fast, reliable coordination mechanism that works across any number of application instances.
Real-World Use
DodaZIP's conversion workers use Redis locks to prevent duplicate file processing. When a worker starts processing a file, it acquires a lock with 5-minute TTL. If the worker crashes, the lock auto-expires and another worker picks up the file. This eliminates duplicate conversions while handling worker failures gracefully.
Basic Distributed Lock
Simple lock using SET NX with expiration:
import redis
import time
import uuid
r = redis.Redis(decode_responses=True)
class SimpleDistributedLock:
def __init__(self, redis_client):
self.r = redis_client
def acquire(self, lock_key, ttl_seconds=30, owner_id=None):
"""Acquire a distributed lock with auto-expiry."""
owner = owner_id or str(uuid.uuid4())
acquired = self.r.set(lock_key, owner, nx=True, ex=ttl_seconds)
if acquired:
return {"acquired": True, "owner": owner, "ttl": ttl_seconds}
return {"acquired": False}
def release(self, lock_key, owner_id):
"""Release the lock only if we still own it (prevents releasing others' locks)."""
script = """
if redis.call("GET", KEYS[1]) == ARGV[1] then
return redis.call("DEL", KEYS[1])
else
return 0
end
"""
result = self.r.eval(script, 1, lock_key, owner_id)
return {"released": result == 1}
def extend(self, lock_key, owner_id, additional_ttl=30):
"""Extend the lock TTL if we still own it."""
script = """
if redis.call("GET", KEYS[1]) == ARGV[1] then
return redis.call("EXPIRE", KEYS[1], ARGV[2])
else
return 0
end
"""
result = self.r.eval(script, 1, lock_key, owner_id, additional_ttl)
return {"extended": result == 1}
def with_lock(self, lock_key, ttl_seconds, fn, *args, **kwargs):
"""Context manager style: acquire lock, run function, release."""
owner = str(uuid.uuid4())
acquired = self.acquire(lock_key, ttl_seconds, owner)
if not acquired["acquired"]:
raise Exception(f"Could not acquire lock: {lock_key}")
try:
return fn(*args, **kwargs)
finally:
self.release(lock_key, owner)
lock = SimpleDistributedLock(r)
result = lock.acquire("lock:job:42", ttl_seconds=30)
print(f"Lock acquired: {result['acquired']}")
result = lock.release("lock:job:42", result['owner'])
print(f"Lock released: {result['released']}")
try:
def expensive_work():
return "Work completed!"
result = lock.with_lock("lock:compute", 30, expensive_work)
print(f"With lock: {result}")
except Exception as e:
print(f"Failed: {e}")
Expected output:
Lock acquired: True
Lock released: True
With lock: Work completed!
Redlock Algorithm
Multi-node lock for stronger safety guarantees:
import redis
import time
import uuid
class Redlock:
def __init__(self, redis_nodes):
"""redis_nodes is a list of Redis connections."""
self.nodes = redis_nodes
self.quorum = len(redis_nodes) // 2 + 1
def acquire(self, resource, ttl_ms=30000):
"""Acquire a Redlock distributed lock across multiple nodes."""
value = str(uuid.uuid4())
start = int(time.time() * 1000)
acquired = 0
for node in self.nodes:
try:
if node.set(resource, value, nx=True, px=ttl_ms):
acquired += 1
except redis.ConnectionError:
continue
elapsed = int(time.time() * 1000) - start
if acquired >= self.quorum and elapsed < ttl_ms:
return {"acquired": True, "value": value, "nodes": acquired}
else:
for node in self.nodes:
try:
node.delete(resource)
except redis.ConnectionError:
continue
return {"acquired": False, "reason": "quorum_not_reached", "nodes": acquired}
def release(self, resource, value):
"""Release the lock by deleting it only if the value matches."""
script = """
if redis.call("GET", KEYS[1]) == ARGV[1] then
return redis.call("DEL", KEYS[1])
else
return 0
end
"""
released = 0
for node in self.nodes:
try:
released += node.eval(script, 1, resource, value)
except redis.ConnectionError:
continue
return released > 0
nodes = [redis.Redis(host="127.0.0.1", port=6379 + i) for i in range(3)]
redlock = Redlock(nodes)
lock = redlock.acquire("resource:payment", ttl_ms=10000)
print(f"Redlock acquired: {lock.get('acquired')}, nodes: {lock.get('nodes')}")
if lock.get("acquired"):
released = redlock.release("resource:payment", lock["value"])
print(f"Redlock released: {released}")
Expected output:
Redlock acquired: True, nodes: 3
Redlock released: True
Lock Renewal Pattern
Keep locks alive during long operations:
import redis
import time
import uuid
import threading
r = redis.Redis(decode_responses=True)
class RenewableLock:
def __init__(self, redis_client):
self.r = redis_client
self._renewal_thread = None
self._running = False
self.owner = None
self.key = None
def acquire(self, key, ttl=30):
"""Acquire a lock and start auto-renewal."""
self.owner = str(uuid.uuid4())
self.key = key
self.ttl = ttl
acquired = self.r.set(key, self.owner, nx=True, ex=ttl)
if not acquired:
return False
self._running = True
self._renewal_thread = threading.Thread(target=self._renew_loop, daemon=True)
self._renewal_thread.start()
return True
def _renew_loop(self):
"""Periodically extend the lock TTL."""
while self._running:
time.sleep(self.ttl * 0.6)
if not self._running:
break
script = """
if redis.call("GET", KEYS[1]) == ARGV[1] then
return redis.call("EXPIRE", KEYS[1], ARGV[2])
else
return 0
end
"""
try:
self.r.eval(script, 1, self.key, self.owner, self.ttl)
except:
break
def release(self):
"""Release the lock and stop renewal."""
self._running = False
if self._renewal_thread:
self._renewal_thread.join(timeout=1)
script = """
if redis.call("GET", KEYS[1]) == ARGV[1] then
return redis.call("DEL", KEYS[1])
else
return 0
end
"""
return self.r.eval(script, 1, self.key, self.owner) == 1
lock = RenewableLock(r)
if lock.acquire("lock:long_job", ttl=10):
print(f"Lock acquired with auto-renewal (TTL={lock.ttl}s)")
time.sleep(5)
remaining_ttl = r.ttl("lock:long_job")
print(f"After 5s, remaining TTL: {remaining_ttl}s (should be ~10s due to renewal)")
lock.release()
print("Lock released")
Expected output:
Lock acquired with auto-renewal (TTL=10s)
After 5s, remaining TTL: 10s (should be ~10s due to renewal)
Lock released
Common Mistakes
- Not setting a TTL on locks — without TTL, a crashed worker holds the lock forever, causing Deadlock. Always set ex/px on lock keys.
- Releasing locks without verifying ownership — calling DEL on a lock that another instance acquired releases their lock. Use the ownership verification Lua script.
- Using too-short TTLs for long operations — if the operation takes longer than the TTL, the lock expires and another worker starts the same work. Set TTL to 2-3x the expected operation duration.
- Ignoring clock drift in Redlock — Redlock assumes synchronized clocks across nodes. In environments with significant clock drift, the algorithm's safety guarantees weaken.
- Not handling lock acquisition failures gracefully — if a lock cannot be acquired, retry with exponential backoff instead of failing immediately.
Practice Questions
- Why must a distributed lock have a TTL?
- What problem does the Lua script for lock release solve?
- How does Redlock achieve safety across multiple Redis nodes?
- When should you use lock renewal during long operations?
- What is a fencing token and why is it needed for strong consistency?
Challenge
Build a distributed task scheduler using Redis locks. Workers acquire locks on available tasks, process them, and release the locks. Tasks have a timeout — if a lock expires, another worker picks up the task. Implement: task locking with owner verification, automatic lock renewal for long tasks, deadlock detection (locks held > 2x TTL), and a retry mechanism for failed tasks.
FAQ
Mini Project
Build a distributed task coordinator using Redis locks. Workers acquire locks on task IDs, process tasks with progress tracking, and auto-renew locks during long operations. Implement: (1) a task queue backed by Redis lists, (2) lock acquisition with retry logic, (3) progress reporting with HSET, (4) automatic deadlock cleanup (locks held > 10 minutes without renewal), and (5) a monitoring endpoint showing active locks, pending tasks, and completed tasks.
What's Next
Continue with Cache Batch Operations to learn efficient bulk cache operations with MSET, MGET, and pipelining. Then explore Cache Pipelining for reducing round-trip latency.
Built by the developers of DodaTech
Doda Browser, DodaZIP & Durga Antivirus Pro