Rate Limiting in API Gateway — Protecting Backends from Traffic Spikes

In this tutorial, you will learn about Rate Limiting in API Gateway. We cover key concepts, practical examples, and best practices to help you master this topic.

Rate limiting in an API gateway controls how many requests a client can send within a specific time window, preventing abuse, ensuring fair resource allocation, and protecting backend services from overload.

What You'll Learn

• Why the gateway is the ideal place for rate limiting
• Token bucket and Sliding Window implementations
• Per-client, per-endpoint, and global rate limit strategies

Why It Matters

Without rate limiting, a single misconfigured client or malicious actor can consume all backend capacity, degrading service for everyone. Placing rate limits at the gateway ensures enforcement before requests reach backend services, saving compute resources and protecting against DDoS attacks.

Real-World Use

Durga Antivirus Pro's partner API allows 1,000 requests per minute per API key. The gateway checks the rate limit on every request before forwarding to the scan service. If a partner exceeds the limit, the gateway returns 429 Too Many Requests with a Retry-After header.

flowchart LR
    Client["Client"] --> RL["Rate Limiter\nin Gateway"]
    RL -->|"Under limit"| Backend["Backend Service"]
    RL -->|"Over limit"| Error["429 Response"]
    style RL fill:#dbeafe,stroke:#2563eb
    style Error fill:#fecaca,stroke:#dc2626

Token Bucket Algorithm

The token bucket is a popular algorithm for API rate limiting. The bucket holds tokens, each representing one request. Tokens refill at a fixed rate, and requests consume tokens. If the bucket is empty, the request is rejected.

import time
import threading

class TokenBucket:
    def __init__(self, rate, capacity):
        self.rate = rate
        self.capacity = capacity
        self.tokens = capacity
        self.last_refill = time.time()
        self.lock = threading.Lock()

    def allow_request(self):
        with self.lock:
            now = time.time()
            elapsed = now - self.last_refill
            self.tokens = min(self.capacity, self.tokens + elapsed * self.rate)
            self.last_refill = now
            if self.tokens >= 1:
                self.tokens -= 1
                return True
            return False

bucket = TokenBucket(rate=10, capacity=20)
for i in range(25):
    allowed = bucket.allow_request()
    if allowed:
        print(f"Request {i+1}: allowed")
    else:
        print(f"Request {i+1}: rate limited")

Expected output (first 20 allowed, last 5 limited):

Request 1: allowed
...
Request 20: allowed
Request 21: rate limited
...
Request 25: rate limited

Sliding Window with Redis

For distributed gateways, use Redis to maintain accurate counters across instances:

import redis
import time

class SlidingWindowRateLimiter:
    def __init__(self, redis_client):
        self.redis = redis_client

    def allow_request(self, client_id, max_requests, window_seconds):
        now = int(time.time())
        window_start = now - window_seconds
        key = f"ratelimit:{client_id}"
        self.redis.zremrangebyscore(key, 0, window_start)
        current_count = self.redis.zcard(key)
        if current_count >= max_requests:
            return False
        self.redis.zadd(key, {str(now): now})
        self.redis.expire(key, window_seconds)
        return True

Gateway Integration

The rate limiter runs as middleware in the gateway pipeline:

from flask import Flask, request, jsonify

app = Flask(__name__)
limiter = TokenBucket(rate=100, capacity=100)

@app.before_request
def check_rate_limit():
    if not limiter.allow_request():
        return jsonify({"error": "Rate limit exceeded"}), 429

@app.route("/api/scan")
def scan():
    return {"status": "scanning"}

Common Mistakes

1. Rate Limiting After Expensive Processing

Check the rate limit as the first step in the gateway, not after processing or logging. Early rejection saves resources.

2. Using Fixed Window Without Offset

Fixed Windows at minute boundaries cause bursts at the window edge. Use sliding window or token bucket for smoother limits.

3. Not Differentiating Client Tiers

All clients should not have the same limit. Implement tiered limits: free=100/hour, pro=1000/hour, enterprise=10000/hour.

4. Forgetting Rate Limit Headers

Clients need to know their limit and remaining requests. Include X-RateLimit-Limit, X-RateLimit-Remaining, and X-RateLimit-Reset.

5. Single Instance Rate Counters

Gateway instances behind a load balancer must share rate limit state, or clients can exceed limits by hitting different instances.

Practice Questions

1. Why is the gateway a better place for rate limiting than the backend service?
2. How does the token bucket algorithm differ from a simple counter?
3. What problem does sliding window solve that fixed window does not?
4. Why must rate limit state be shared across gateway instances?
5. What headers should a rate-limited API return?

Answers:

1. The gateway rejects excess requests before they consume backend resources, protecting all backend services from abuse.
2. Token bucket allows bursts up to the bucket capacity while maintaining an average rate; a simple counter has rigid boundaries.
3. Sliding window prevents burst traffic at window boundaries that fixed window allows at the start/end of each window.
4. Without shared state, a client can exceed the limit by distributing requests across multiple gateway instances.
5. X-RateLimit-Limit (max requests), X-RateLimit-Remaining (requests left), X-RateLimit-Reset (window reset time), and Retry-After when limited.

Challenge: Design a rate limiting Strategy for a multi-tenant API with free, pro, and enterprise tiers where pro users get 10x the limit of free users.

FAQ

Mini Project

Build a rate-limited gateway with three tiers: anonymous (10 req/min), basic (100 req/min), and premium (1000 req/min). Identify the tier from the API key in the request header. Use a sliding window algorithm with Redis.

What's Next

Continue with Authentication in API Gateway to secure your gateway, or explore SSL Termination in Gateway for TLS management.