Distributed Rate Limiting — Complete Implementation Guide
In this tutorial, you will learn about Distributed Rate Limiting. We cover key concepts, practical examples, and best practices to help you master this topic.
Distributed rate limiting ensures consistent request limits across multiple application servers, preventing a client from exceeding limits by switching between server instances.
What You'll Learn
By the end of this tutorial, you will implement distributed rate limiting with Redis, understand Consistency Models, and handle edge cases like clock skew and network partitions.
Why It Matters
Without distributed rate limiting, each server tracks limits independently. A client can make N requests per server per window, multiplying their effective limit by the number of servers.
Real-World Use
DodaTech's API runs on 10 Kubernetes pods. Distributed Redis rate limiting ensures users get exactly 100 requests per minute regardless of which pod handles their request.
Distributed Rate Limiting Learning Path
flowchart LR
A[Redis Rate Limiting] --> B[Distributed Rate Limiting]
B --> C[Centralized Redis]
B --> D[Consistency Models]
B --> E{You Are Here}
style E fill:#f90,color:#fff
The Problem with Per-Server Limiting
With in-memory rate limiting, each server maintains its own counter. A client can exceed the intended limit by distributing requests across servers.
// Problem: 3 servers, limit 100 req/min each
// Client can make 300 req/min by sending 100 to each server
// Correct distributed limit: 100 req/min total across all servers
Centralized Redis Solution
A centralized Redis store provides a single source of truth for all rate limit counters, ensuring consistent limits across all servers.
const Redis = require("ioredis");
const redis = new Redis(process.env.REDIS_URL);
async function distributedLimiter(req, res, next) {
const key = `dlimit:${req.ip}:${Math.floor(Date.now() / 60000)}`;
const max = 100;
const count = await redis.incr(key);
if (count === 1) {
await redis.expire(key, 120);
}
if (count > max) {
return res.status(429).json({
error: "Rate limit exceeded",
global: true
});
}
next();
}
Eventual Consistency Approach
For high-traffic systems where atomic consistency is not critical, an eventual consistency approach uses local counters that periodically sync to Redis.
class EventuallyConsistentLimiter {
constructor(redis, options) {
this.redis = redis;
this.localCounts = new Map();
this.windowMs = options.windowMs || 60000;
this.max = options.max || 100;
this.localBuffer = options.localBuffer || Math.floor(this.max * 0.1);
this.syncInterval = setInterval(() => this.sync(), 5000);
}
async check(key) {
const now = Date.now();
const windowKey = Math.floor(now / this.windowMs);
const localKey = `${key}:${windowKey}`;
let localCount = this.localCounts.get(localKey) || 0;
localCount++;
this.localCounts.set(localKey, localCount);
if (localCount <= this.max - this.localBuffer) {
return { allowed: true };
}
return this.checkRemote(key, windowKey);
}
async checkRemote(key, windowKey) {
const remoteKey = `elimit:${key}:${windowKey}`;
const total = await this.redis.incr(remoteKey);
if (total === 1) {
await this.redis.expire(remoteKey, 120);
}
return { allowed: total <= this.max };
}
async sync() {
const now = Date.now();
const windowKey = Math.floor(now / this.windowMs);
for (const [key, count] of this.localCounts) {
if (key.endsWith(`:${windowKey}`) && count > 0) {
const baseKey = key.replace(`:${windowKey}`, "");
const remoteKey = `elimit:${baseKey}:${windowKey}`;
await this.redis.incrby(remoteKey, count);
this.localCounts.set(key, 0);
}
}
}
}
Clock Skew Handling
Multi-region deployments suffer from clock skew. Synchronized clocks are essential for accurate rate limiting.
class ClockSkewAwareLimiter {
constructor(redis) {
this.redis = redis;
this.clockOffset = 0;
this.init();
}
async init() {
try {
const redisTime = await this.redis.time();
const redisTimestamp = parseInt(redisTime[0]) * 1000 +
Math.floor(parseInt(redisTime[1]) / 1000);
this.clockOffset = redisTimestamp - Date.now();
} catch {
this.clockOffset = 0;
}
}
now() {
return Date.now() + this.clockOffset;
}
getWindowId(timestamp) {
return Math.floor(timestamp / 60000);
}
async check(key) {
const windowId = this.getWindowId(this.now());
const redisKey = `skew:${key}:${windowId}`;
const count = await this.redis.incr(redisKey);
if (count === 1) await this.redis.expire(redisKey, 120);
return count <= 100;
}
}
Common Mistakes
Not using atomic operations -- GET + SET race conditions cause inconsistent counts. Always use INCR or Lua scripts.
Ignoring network latency -- Redis calls add 1-5ms latency per request. Batch operations where possible.
Not handling Redis failover -- When Redis is down, rate limiting stops. Implement local fallback with a reduced capacity.
Using different Redis instances per region -- Each region has its own counter. Use global Redis or CRDT-based approaches.
Not monitoring rate limit accuracy -- Compare local vs distributed counts. Large discrepancies indicate bugs.
Practice Questions
Why does per-server rate limiting fail in distributed deployments? A client can distribute requests across servers, multiplying their effective limit by the number of servers.
What is the advantage of eventual consistency for rate limiting? Lower latency (local counters) with periodic sync. Acceptable for non-critical rate limits where slight overages are tolerable.
How does clock skew affect fixed window rate limiting? Servers with different clocks calculate different window boundaries, causing inconsistent limits.
Challenge: Design a rate limiting system that works across multiple data centers.
// Use a global Redis cluster for shared counters
// Each region has a local cache that syncs to global Redis
// Use CRDT counters for conflict-free merging across regions
FAQ
Mini Project
Build a distributed rate limiter with Redis, clock skew handling, local cache for performance, and graceful degradation during Redis outages.
const Redis = require("ioredis");
class ProductionDistributedLimiter {
constructor(redisUrl) {
this.redis = new Redis(redisUrl);
this.local = new Map();
this.syncTimer = setInterval(() => this.flushLocal(), 10000);
}
async allow(key, limit = 100, windowMs = 60000) {
const window = Math.floor(Date.now() / windowMs);
const localKey = `${key}:${window}`;
const local = (this.local.get(localKey) || 0) + 1;
this.local.set(localKey, local);
if (local <= limit * 0.8) {
return { allowed: true, source: "local" };
}
try {
const redisKey = `prod:${key}:${window}`;
const remote = await this.redis.incr(redisKey);
if (remote === 1) await this.redis.expire(redisKey, Math.ceil(windowMs / 1000) * 2);
return { allowed: remote <= limit, source: "redis", count: remote };
} catch {
return { allowed: local <= limit, source: "fallback" };
}
}
async flushLocal() {
const now = Math.floor(Date.now() / 60000);
for (const [key, count] of this.local) {
if (key.endsWith(`:${now}`) && count > 0) {
const baseKey = key.replace(`:${now}`, "");
try {
const redisKey = `prod:${baseKey}:${now}`;
await this.redis.incrby(redisKey, count);
this.local.set(key, 0);
} catch {}
}
}
}
}
What's Next
Now that you understand distributed rate limiting, explore proper rate limit HTTP headers. Then learn about advanced rate limiting patterns and strategies.
Built by the developers of DodaTech
Doda Browser, DodaZIP & Durga Antivirus Pro