API Gateway Project — Build a Complete Gateway from Scratch

In this tutorial, you will learn about API Gateway Project. We cover key concepts, practical examples, and best practices to help you master this topic.

Build a complete API gateway that combines routing, JWT authentication, Redis-based rate limiting, response caching, structured logging, circuit breaking, and WebSocket support into a single production-ready system.

What You'll Build

• Flask gateway with plugin-style middleware pipeline
• Redis-backed rate limiting and caching
• JWT authentication with public key verification
• Circuit breaker for downstream services
• Nginx as the edge layer for TLS and static content

Why This Project Matters

Individual gateway features are useful, but combining them into a cohesive system reveals integration challenges: middleware ordering, error propagation, logging consistency, and configuration management. This project gives you hands-on experience building a real gateway.

flowchart LR
    Client["Client"] --> Nginx["Nginx Edge\nTLS + Static"]
    Nginx --> Flask["Flask Gateway\nRouting + Auth + RL"]
    Flask --> Auth["JWT Auth\nMiddleware"]
    Flask --> RL["Rate Limit\nMiddleware"]
    Flask --> Cache["Cache\nMiddleware"]
    Flask --> CB["Circuit Breaker\nMiddleware"]
    CB --> S1["Service 1"]
    CB --> S2["Service 2"]
    style Flask fill:#dbeafe,stroke:#2563eb

Project Structure

api-gateway/
  nginx/
    nginx.conf
  gateway/
    app.py
    middleware/
      auth.py
      rate_limit.py
      cache.py
      circuit_breaker.py
      logging_middleware.py
    config.py
    requirements.txt
  docker-compose.yml

Step 1: Configuration

# config.py
import os

class Config:
    REDIS_URL = os.getenv("REDIS_URL", "redis://redis:6379/0")
    JWT_PUBLIC_KEY = open("/etc/gateway/jwt-public.pem").read()
    JWT_ALGORITHM = "RS256"
    JWT_AUDIENCE = "api.dodatech.com"
    RATE_LIMIT_DEFAULT = 100
    RATE_LIMIT_WINDOW = 60
    CACHE_TTL = 300
    CIRCUIT_BREAKER_THRESHOLD = 5
    CIRCUIT_BREAKER_TIMEOUT = 30
    UPSTREAM_SERVICES = {
        "users": "http://user-service:8080",
        "orders": "http://order-service:8080",
        "products": "http://product-service:8080",
    }

Step 2: Logging Middleware

# middleware/logging_middleware.py
import uuid
import time
import logging

logger = logging.getLogger("gateway")

class LoggingMiddleware:
    def __init__(self, app):
        self.app = app

    def __call__(self, environ, start_response):
        trace_id = str(uuid.uuid4())
        environ["trace_id"] = trace_id
        start_time = time.time()

        def custom_start_response(status, headers, exc_info=None):
            duration = time.time() - start_time
            logger.info({
                "trace_id": trace_id,
                "method": environ.get("REQUEST_METHOD"),
                "path": environ.get("PATH_INFO"),
                "status": status,
                "duration_ms": round(duration * 1000, 2),
            })
            headers.append(("X-Trace-ID", trace_id))
            return start_response(status, headers, exc_info)

        return self.app(environ, custom_start_response)

Step 3: Gateway Application

# app.py
from flask import Flask, request, jsonify
from middleware.auth import AuthMiddleware
from middleware.rate_limit import RateLimitMiddleware
from middleware.cache import CacheMiddleware
from middleware.circuit_breaker import CircuitBreakerMiddleware
import requests

app = Flask(__name__)
app.wsgi_app = LoggingMiddleware(app.wsgi_app)

SERVICES = {
    "users": "http://user-service:8080",
    "orders": "http://order-service:8080",
    "products": "http://product-service:8080",
}

circuit_breakers = {
    name: CircuitBreaker(threshold=5, timeout=30)
    for name in SERVICES
}

@app.route("/api/<service>/<path:subpath>", methods=["GET", "POST", "PUT", "DELETE"])
def gateway_route(service, subpath):
    if service not in SERVICES:
        return jsonify({"error": "Unknown service"}), 404

    backend_url = f"{SERVICES[service]}/{subpath}"

    def call_backend():
        return requests.request(
            method=request.method,
            url=backend_url,
            params=request.args,
            json=request.get_json(silent=True),
            headers={k: v for k, v in request.headers if k.lower() not in ("host",)},
        )

    cb = circuit_breakers[service]
    result = cb.call(call_backend)

    if result is None:
        return jsonify({
            "error": f"{service} service unavailable",
            "trace_id": request.environ.get("trace_id")
        }), 503

    return (result.content, result.status_code, result.headers.items())

Step 4: Docker Compose

# docker-compose.yml
version: "3.8"
services:
  nginx:
    image: nginx:alpine
    ports:
      - "443:443"
      - "80:80"
    volumes:
      - ./nginx/nginx.conf:/etc/nginx/nginx.conf

  gateway:
    build: ./gateway
    environment:
      - REDIS_URL=redis://redis:6379/0

  redis:
    image: redis:7-alpine

  user-service:
    image: your-user-service
  order-service:
    image: your-order-service
  product-service:
    image: your-product-service

Step 5: Test the Gateway

# Test routing
curl -s http://localhost/api/users/me \
  -H "Authorization: Bearer $JWT"

# Test rate limiting
for i in $(seq 1 120); do
  curl -s -o /dev/null -w "%{http_code}\n" \
    http://localhost/api/products \
    -H "Authorization: Bearer $JWT"
done

# Test circuit breaker
# Stop the user-service and observe 503 responses

Common Mistakes

1. Middleware Ordering

Run logging first, then auth, then rate limiting, then circuit breaker, then caching, then routing. Wrong ordering causes auth failures to be cached or rate limits to apply before authentication.

2. Shared State in Multi-Process Gateways

In-memory rate limit counters break across processes. Always use Redis for shared state.

3. Not Handling Gateway Startup Order

The gateway starts before Redis or upstream services are ready. Add retry logic for initial connections.

4. Missing Graceful Shutdown

When the gateway stops, in-flight requests get dropped. Implement SIGTERM handling with a drain period.

5. No Configuration Validation

Invalid config crashes the gateway at runtime. Validate configuration at startup with clear error messages.

Practice Questions

1. Why should logging run first in the middleware pipeline?
2. How does Redis help share state across gateway instances?
3. What is the purpose of a drain period during gateway shutdown?
4. Why must middleware ordering be carefully designed?
5. How do you test the circuit breaker behavior in this project?

Answers:

1. Logging captures every request including those rejected by auth or rate limiting, providing full observability.
2. Redis stores rate limit counters, cache entries, and circuit breaker state that all gateway instances can access consistently.
3. A drain period stops accepting new requests but finishes processing in-flight requests, preventing data loss.
4. Incorrect ordering can cause security bypasses (auth after routing) or functional issues (caching error responses).
5. Stop the backend service and send requests. The circuit breaker opens after N failures and returns 503. Restart the backend and verify recovery.

Challenge: Extend this project to include a developer portal, API key management, and a metrics dashboard with Prometheus and Grafana integration.

FAQ

Mini Project

Complete the full gateway project with all middleware components. Deploy with Docker Compose and demonstrate the following scenarios: successful routing, authentication rejection, rate limit exceeded, circuit breaker open, cached response, and WebSocket forwarding.

What's Next

Review the API Gateway Introduction to reinforce core concepts, or explore Rate Limiting Complete Guide for a deep dive into rate limiting algorithms and strategies.