Skip to content

Rate Limiting Project — Build a Complete Rate Limiting System

DodaTech Updated 2026-06-28 7 min read

In this tutorial, you will learn about Rate Limiting Project. We cover key concepts, practical examples, and best practices to help you master this topic.

Build a complete rate limiting system that combines fixed window, sliding window, and token bucket algorithms with Redis persistence, tiered per-user limits, rate limit headers, and a resilient test client with exponential backoff.

What You'll Build

  • Flask API with configurable rate limiting algorithms
  • Redis-backed counters and token buckets
  • Tiered limits (free/pro/enterprise)
  • Rate limit headers on every response
  • Test client with exponential backoff

Why This Project Matters

Individual rate limiting components are useful, but integrating them into a cohesive system reveals challenges: algorithm selection per endpoint, header consistency, cache invalidation, and testing under load.

flowchart LR
    Client["Client"] --> Router["Flask Router"]
    Router -->|"/api/scan"| TB["Token Bucket\nRate=10/s, Burst=50"]
    Router -->|"/api/reports"| SW["Sliding Window\n1000 req/hour"]
    Router -->|"/api/login"| FW["Fixed Window\n5 req/min"]
    TB --> Redis["Redis"]
    SW --> Redis
    FW --> Redis
    Redis --> Headers["Rate Limit\nHeaders"]
    style Router fill:#dbeafe,stroke:#2563eb
    style Redis fill:#bbf7d0,stroke:#16a34a

Project Structure

rate-limiting-system/
  app.py
  limiters/
    __init__.py
    fixed_window.py
    sliding_window.py
    token_bucket.py
  middleware.py
  config.py
  client.py
  requirements.txt

Step 1: Configuration

# config.py
import os

TIERS = {
    "free": {
        "scan": {"algorithm": "token_bucket", "rate": 1, "capacity": 10},
        "reports": {"algorithm": "sliding_window", "limit": 100, "window": 3600},
        "login": {"algorithm": "fixed_window", "limit": 5, "window": 60},
    },
    "pro": {
        "scan": {"algorithm": "token_bucket", "rate": 10, "capacity": 100},
        "reports": {"algorithm": "sliding_window", "limit": 1000, "window": 3600},
        "login": {"algorithm": "fixed_window", "limit": 50, "window": 60},
    },
    "enterprise": {
        "scan": {"algorithm": "token_bucket", "rate": 100, "capacity": 500},
        "reports": {"algorithm": "sliding_window", "limit": 10000, "window": 3600},
        "login": {"algorithm": "fixed_window", "limit": 200, "window": 60},
    },
}

REDIS_URL = os.getenv("REDIS_URL", "redis://localhost:6379/0")

Step 2: Rate Limiter Implementations

# limiters/token_bucket.py
import time
import json
import redis

class TokenBucket:
    def __init__(self, redis_client):
        self.redis = redis_client

    def allow_request(self, client_id, rate, capacity):
        key = f"tb:{client_id}"
        now = time.time()

        lua = """
        local key = KEYS[1]
        local now = tonumber(ARGV[1])
        local rate = tonumber(ARGV[2])
        local capacity = tonumber(ARGV[3])

        local data = redis.call('GET', key)
        if data then
            local tokens, last_refill = unpack(cjson.decode(data))
            local elapsed = now - last_refill
            tokens = math.min(capacity, tokens + elapsed * rate)
            if tokens >= 1 then
                tokens = tokens - 1
                redis.call('SET', key, cjson.encode({tokens, now}), 'EX', 3600)
                return {1, tokens, now + (1 / rate)}
            else
                return {0, tokens, now + (1 / rate)}
            end
        else
            local tokens = capacity - 1
            redis.call('SET', key, cjson.encode({tokens, now}), 'EX', 3600)
            return {1, tokens, now}
        end
        """
        result = self.redis.eval(lua, 1, key, now, rate, capacity)
        return {
            "allowed": result[0] == 1,
            "remaining": int(result[1]),
            "reset": int(result[2]),
        }

# limiters/sliding_window.py
import time

class SlidingWindow:
    def __init__(self, redis_client):
        self.redis = redis_client

    def allow_request(self, client_id, limit, window):
        key = f"sw:{client_id}"
        now = time.time()
        cutoff = now - window

        lua = """
        local key = KEYS[1]
        local cutoff = ARGV[1]
        local now = ARGV[2]
        local limit = tonumber(ARGV[3])
        local window = tonumber(ARGV[4])

        redis.call('ZREMRANGEBYSCORE', key, 0, cutoff)
        local count = redis.call('ZCARD', key)

        if count < limit then
            redis.call('ZADD', key, now, now)
            redis.call('EXPIRE', key, window * 2)
            return {1, limit - count - 1, cutoff + window}
        end
        return {0, 0, cutoff + window}
        """
        result = self.redis.eval(lua, 1, key, cutoff, now, limit, window)
        return {
            "allowed": result[0] == 1,
            "remaining": int(result[1]),
            "reset": int(result[2]),
        }

Step 3: Flask Middleware

# middleware.py
from flask import request, jsonify, g
from limiters.token_bucket import TokenBucket
from limiters.sliding_window import SlidingWindow
from limiters.fixed_window import FixedWindow
from config import TIERS

class RateLimitMiddleware:
    def __init__(self, redis_client):
        self.redis = redis_client
        self.limiters = {
            "token_bucket": TokenBucket(redis_client),
            "sliding_window": SlidingWindow(redis_client),
            "fixed_window": FixedWindow(redis_client),
        }

    def check(self, client_id, tier, endpoint):
        if tier not in TIERS:
            tier = "free"

        endpoint_config = TIERS[tier].get(endpoint, TIERS["free"]["reports"])
        limiter = self.limiters[endpoint_config["algorithm"]]
        config = {k: v for k, v in endpoint_config.items() if k != "algorithm"}
        result = limiter.allow_request(client_id, **config)

        g.rate_limit = result
        return result["allowed"]

Step 4: Flask Application

# app.py
from flask import Flask, request, jsonify, g
from middleware import RateLimitMiddleware
import redis

app = Flask(__name__)
redis_client = redis.Redis.from_url("redis://localhost:6379/0")
rate_limiter = RateLimitMiddleware(redis_client)

def get_client_id():
    api_key = request.headers.get("X-API-Key")
    if api_key:
        return f"apikey:{api_key}"
    return f"ip:{request.remote_addr}"

def get_tier():
    api_key = request.headers.get("X-API-Key")
    tiers = {"sk-free-": "free", "sk-pro-": "pro", "sk-ent-": "enterprise"}
    for prefix, tier in tiers.items():
        if api_key and api_key.startswith(prefix):
            return tier
    return "free"

@app.before_request
def rate_limit():
    client_id = get_client_id()
    tier = get_tier()
    endpoint = request.path.rstrip("/")

    allowed = rate_limiter.check(client_id, tier, endpoint)
    if not allowed:
        return jsonify({
            "error": "Rate limit exceeded",
            "retry_after": int(g.rate_limit.get("reset", time.time() + 60) - time.time())
        }), 429

@app.after_request
def add_headers(response):
    if hasattr(g, "rate_limit"):
        rl = g.rate_limit
        response.headers["X-RateLimit-Limit"] = str(rl.get("limit", 0))
        response.headers["X-RateLimit-Remaining"] = str(rl.get("remaining", 0))
        response.headers["X-RateLimit-Reset"] = str(rl.get("reset", 0))
        response.headers["X-RateLimit-Tier"] = get_tier()
    return response

@app.route("/api/scan")
def scan():
    return jsonify({"status": "scanning"})

@app.route("/api/reports")
def reports():
    return jsonify({"reports": []})

@app.route("/api/login", methods=["POST"])
def login():
    return jsonify({"token": "mock-jwt"})

Step 5: Test Client

# client.py
import requests
import time
import random

class ResilientClient:
    def __init__(self, base_url, api_key, max_retries=5):
        self.base_url = base_url
        self.api_key = api_key
        self.max_retries = max_retries

    def request(self, endpoint):
        for attempt in range(self.max_retries + 1):
            resp = requests.get(
                f"{self.base_url}{endpoint}",
                headers={"X-API-Key": self.api_key}
            )

            remaining = int(resp.headers.get("X-RateLimit-Remaining", 0))
            if remaining <= 5:
                delay = random.uniform(0.5, 1.5)
                print(f"Pacing: {remaining} remaining, waiting {delay:.1f}s")
                time.sleep(delay)
                return self.request(endpoint)

            if resp.status_code == 429:
                retry_after = int(resp.headers.get("Retry-After", 60))
                jitter = random.uniform(0.8, 1.2)
                wait = retry_after * jitter
                print(f"429 on attempt {attempt+1}. Waiting {wait:.1f}s")
                time.sleep(wait)
                continue

            return resp

        return resp

# Test
client = ResilientClient("http://localhost:5000", "sk-free-test")
for i in range(20):
    resp = client.request("/api/scan")
    print(f"Request {i+1}: {resp.status_code}")

Common Mistakes

1. One Algorithm for All Endpoints

Login endpoints need strict limits (5 req/min). Scan endpoints benefit from bursts. Use different algorithms per endpoint.

2. No Graceful Degradation

If Redis fails, rate limiting should degrade gracefully. Fall back to in-memory or allow all requests with logging.

3. Not Testing Under Load

Rate limiting logic often has race conditions that only appear under concurrent load. Test with multiple clients simultaneously.

4. Forgetting Rate Limit Headers on 429

Even rejected requests should include reset time. Clients need to know when to retry.

5. No Monitoring on Rate Limit Metrics

Without monitoring, you cannot tune limits. Track rate limit hit rates, algorithm performance, and false positives.

Practice Questions

  1. Why might different endpoints need different rate limiting algorithms?
  2. How should the system degrade if Redis becomes unavailable?
  3. Why is concurrency testing important for rate limiting?
  4. What metrics should you monitor for rate limiting?
  5. How do you choose between algorithms for different use cases?

Answers:

  1. Login endpoints need strict per-IP limits. Scan endpoints need burst allowance. Reports endpoints need accurate hourly counts.
  2. Fall back to in-memory counters with reduced limits, or allow all requests with a warning log. Reconnect to Redis automatically.
  3. Race conditions in token consumption or counter increments only appear when multiple requests arrive simultaneously.
  4. Rate limit hit rate (429s per minute), algorithm latency, remaining quota distribution, and false positive rate.
  5. Token bucket for bursty endpoints, fixed window for simple limits, sliding window for accuracy, sliding log for compliance.

Challenge: Extend this project to include a Grafana dashboard showing rate limit metrics (429 count, remaining quota distribution, algorithm latency). Add load testing with Locust or k6 to verify correctness under concurrent traffic.

FAQ

How do you determine the right rate limit values?

: Start with conservative estimates based on your infrastructure capacity. Monitor usage for 2 weeks. Set limits at 3x the 95th percentile.

Should rate limits be the same across all regions?

: No. Regional infrastructure capacity varies. Use different limits per region based on local capacity.

How do you communicate rate limit changes to partners?

: Use the rate limit headers (they reflect current limits automatically). Send email notifications for significant changes. Provide a changelog.

Can this system handle 1 million requests per second?

: Redis handles ~100K ops/sec per node. For 1M RPS, use Redis Cluster with Sharding or a local cache with periodic sync.

How often should rate limit configurations be updated?

: Review monthly for the first 3 months, then quarterly. Adjust based on traffic growth and usage patterns.

Mini Project

Complete the full rate limiting system with Redis. Deploy with Docker Compose. Test all three algorithms with a load testing script. Verify that rate limit headers are correct, 429 responses include Retry-After, and the system recovers after Redis restart.

What's Next

Review the Rate Limiting Introduction to reinforce core concepts, or explore API Gateway Complete Guide for integrating rate limiting with gateway features like authentication and Caching.

← Previous Retry-After Header and Backoff Strategies — Client-Side Rate Limit Handling Next → Endpoint-Based Rate Limiting — Per-Route API Traffic Control

Built by the developers of DodaTech

Doda Browser, DodaZIP & Durga Antivirus Pro

Home Browse Rate Limiting