WebSocket Gateway Support — Real-Time Communication in API Gateways

In this tutorial, you will learn about WebSocket Gateway Support. We cover key concepts, practical examples, and best practices to help you master this topic.

WebSocket gateway support enables real-time bidirectional communication between clients and servers through the API gateway, handling the HTTP upgrade handshake, persistent connections, and message routing to appropriate backend services.

What You'll Learn

• How WebSocket upgrades work through a gateway
• Connection management and sticky sessions for WebSocket
• Broadcasting messages and handling disconnections

Why It Matters

Standard API gateways handle HTTP request-response cycles. WebSocket requires persistent connections, protocol upgrades, and connection affinity. A WebSocket-aware gateway manages these complexities, routing real-time traffic alongside regular HTTP APIs.

Real-World Use

Durga Antivirus Pro's real-time threat dashboard uses WebSocket through the gateway. When a new threat is detected worldwide, the backend pushes an alert through the gateway to all connected dashboard clients. The gateway manages thousands of concurrent WebSocket connections across multiple backend instances.

flowchart LR
    Client1["Client 1"] --> GW["Gateway\nWebSocket Aware"]
    Client2["Client 2"] --> GW
    Client3["Client 3"] --> GW
    GW --> WS1["WebSocket Server 1"]
    GW --> WS2["WebSocket Server 2"]
    style GW fill:#dbeafe,stroke:#2563eb

WebSocket Upgrade in Nginx

map $http_upgrade $connection_upgrade {
    default  upgrade;
    ""       close;
}

server {
    listen 80;
    server_name ws.dodatech.com;

    location /ws/ {
        proxy_pass http://websocket_backend;
        proxy_http_version 1.1;
        proxy_set_header Upgrade $http_upgrade;
        proxy_set_header Connection $connection_upgrade;
        proxy_set_header Host $host;
        proxy_read_timeout 86400s;
    }
}

The proxy_read_timeout 86400s prevents Nginx from closing idle WebSocket connections.

WebSocket Proxy in Python

import asyncio
import websockets
from flask import Flask, request

app = Flask(__name__)

BACKEND_WS = "ws://ws-backend:8765"

@app.route("/ws/dashboard")
def websocket_proxy():
    if request.headers.get("Upgrade", "").lower() != "websocket":
        return {"error": "WebSocket connection required"}, 426
    # Gateway passes the upgrade through
    return "", 101

For a full proxy using the websockets library:

import asyncio
import websockets

async def proxy_websocket(client_ws, backend_url):
    async with websockets.connect(backend_url) as backend_ws:
        async def forward_client_to_backend():
            async for message in client_ws:
                await backend_ws.send(message)

        async def forward_backend_to_client():
            async for message in backend_ws:
                await client_ws.send(message)

        await asyncio.gather(
            forward_client_to_backend(),
            forward_backend_to_client()
        )

Connection Affinity with Sticky Sessions

WebSocket connections must stick to the same backend instance. Use IP hash Load Balancing:

upstream websocket_backend {
    ip_hash;
    server ws-srv-1:8765;
    server ws-srv-2:8765;
    server ws-srv-3:8765;
}

Without ip_hash, requests may route to different backends, breaking the WebSocket session.

Health Checks for WebSocket Connections

async def health_check():
    while True:
        for ws_id, ws in list(connections.items()):
            try:
                pong = await asyncio.wait_for(ws.ping(), timeout=5)
            except:
                print(f"Removing dead connection: {ws_id}")
                del connections[ws_id]
        await asyncio.sleep(30)

Common Mistakes

1. Short Proxy Timeouts

WebSocket connections can last hours. Setting proxy_read_timeout to the default 60s disconnects idle connections. Set to 24h or more.

2. No Connection Affinity

Round-robin load balancing sends WebSocket upgrade requests to different backends, breaking the connection. Use IP hash or sticky cookies.

3. Not Handling Backend Disconnections

When a WebSocket backend dies, all its connections drop. The gateway should attempt reconnection or notify clients.

4. Mixing HTTP and WebSocket on the Same Route

A route configured for WebSocket may fail for regular HTTP requests. Use separate path prefixes or handle both upgrade and non-upgrade cases.

5. No Rate Limiting on Messages

Without message rate limiting, a client can flood the WebSocket with messages. Implement per-connection message quotas.

Practice Questions

1. How does a WebSocket connection start and what role does the gateway play?
2. Why must the gateway set proxy_read_timeout to a high value for WebSocket?
3. What is connection affinity and why is it critical for WebSocket?
4. How does the gateway handle a WebSocket backend that goes down?
5. Why should WebSocket connections be rate limited?

Answers:

1. The client sends an HTTP upgrade request. The gateway forwards the upgrade headers to the backend, establishing a persistent bidirectional channel.
2. WebSocket connections can remain idle for minutes or hours. A low timeout disconnects idle connections prematurely.
3. Connection affinity ensures all messages from a client route to the same backend instance, preserving the WebSocket session state.
4. The gateway detects the disconnection, closes the client connection, and optionally attempts reconnection or returns an error.
5. Without rate limiting, a single client sending thousands of messages per second can overwhelm the backend and other clients.

Challenge: Design a WebSocket gateway architecture for a real-time chat application with 10,000 concurrent users. Include connection affinity, rate limiting (10 messages/sec per user), and reconnection handling.

FAQ

Mini Project

Build a WebSocket gateway using Python FastAPI that proxies connections to a backend WebSocket server. Implement sticky session routing using IP hash, set a 24-hour timeout, add message rate limiting (5 messages/sec per connection), and handle backend disconnections gracefully.

What's Next

Continue with API Gateway Project to build a complete gateway from scratch, combining all concepts learned in this series.