Gateway Clustering — High Availability and Horizontal Scaling
In this tutorial, you'll learn about Gateway Clustering. We cover key concepts, practical examples, and best practices to help you understand and apply this topic effectively.
Gateway clustering enables multiple gateway instances to work together as a single logical entry point, providing high availability and horizontal scalability.
What You'll Learn
By the end of this lesson, you will implement gateway clustering with shared state, configure leader election for coordinated tasks, set up health-based routing, and deploy zero-downtime updates.
Why It Matters
A single gateway instance is a single point of failure. Clustering eliminates this risk and allows the gateway to handle increasing traffic by adding more instances.
Real-World Use
Durga Antivirus Pro runs a 3-node gateway cluster behind a load balancer, with Redis-backed shared rate limiting and session state, surviving individual node failures without service interruption.
Gateway Cluster Architecture
flowchart TD
LB[Load Balancer]-->GW1[Gateway Node 1]
LB-->GW2[Gateway Node 2]
LB-->GW3[Gateway Node 3]
GW1-->Redis[(Redis - Shared State)]
GW2-->Redis
GW3-->Redis
GW1-->Backend1
GW2-->Backend2
GW3-->Backend3
Cluster Node Discovery
Automatically discover and register gateway nodes in the cluster.
import time
import json
import uuid
from typing import Dict, List, Optional, Set
from threading import Thread
import socket
class ClusterNode:
def __init__(self, node_id: str,
host: str, port: int,
health_endpoint: str = "/health"):
self.node_id = node_id
self.host = host
self.port = port
self.health_endpoint = health_endpoint
self.status: str = "active"
self.last_heartbeat: float = time.time()
self.load: float = 0.0
class ClusterDiscovery:
def __init__(self, redis_client=None):
self.this_node_id = str(uuid.uuid4())
self.nodes: Dict[str, ClusterNode] = {}
self.redis = redis_client
def register(self, host: str, port: int):
node = ClusterNode(
self.this_node_id, host, port
)
self.nodes[self.this_node_id] = node
if self.redis:
self.redis.hset(
"gateway:cluster:nodes",
self.this_node_id,
json.dumps({
"host": host,
"port": port,
"status": "active",
"registered": time.time()
})
)
def discover_peers(self) -> List[ClusterNode]:
if self.redis:
raw_nodes = self.redis.hgetall(
"gateway:cluster:nodes"
)
for node_id, data in raw_nodes.items():
if node_id.decode() != self.this_node_id:
info = json.loads(data)
node = ClusterNode(
node_id.decode(),
info["host"],
info["port"]
)
self.nodes[node.node_id] = node
return list(self.nodes.values())
def get_active_nodes(self) -> List[ClusterNode]:
now = time.time()
return [
n for n in self.nodes.values()
if (n.status == "active"
and now - n.last_heartbeat < 30)
]
def leave_cluster(self):
if self.redis:
self.redis.hdel(
"gateway:cluster:nodes",
self.this_node_id
)
discovery = ClusterDiscovery()
discovery.register("192.168.1.10", 8080)
peers = discovery.discover_peers()
print(f"Discovered {len(peers)} peers")
Leader Election for Coordinated Tasks
Use leader election to assign Singleton tasks like configuration reloads.
import time
from typing import Optional, Callable
import threading
class LeaderElection:
def __init__(self, node_id: str,
redis_client=None,
ttl: int = 30):
self.node_id = node_id
self.redis = redis_client
self.ttl = ttl
self.is_leader = False
self._renewal_thread: Optional[threading.Thread] = None
self._running = False
def try_acquire(self) -> bool:
if self.redis:
acquired = self.redis.setnx(
"gateway:cluster:leader",
self.node_id
)
if acquired:
self.redis.expire(
"gateway:cluster:leader",
self.ttl
)
return bool(acquired)
return True
def start(self):
if self.try_acquire():
self.is_leader = True
self._running = True
self._renewal_thread = threading.Thread(
target=self._renew_leadership,
daemon=True
)
self._renewal_thread.start()
def _renew_leadership(self):
while self._running:
time.sleep(self.ttl // 3)
if self.redis:
ttl = self.redis.ttl("gateway:cluster:leader")
if ttl < self.ttl // 2:
self.redis.expire(
"gateway:cluster:leader",
self.ttl
)
def resign(self):
self._running = False
if self.redis and self.is_leader:
current = self.redis.get("gateway:cluster:leader")
if current and current.decode() == self.node_id:
self.redis.delete("gateway:cluster:leader")
self.is_leader = False
election = LeaderElection("node-1")
if election.try_acquire():
print("This node is the leader")
election.start()
Health-Based Routing
Route traffic away from unhealthy cluster nodes.
from typing import Dict, List, Optional, Tuple
import time
import requests
class HealthRouter:
def __init__(self, check_interval: int = 10):
self.healthy_nodes: Dict[str, bool] = {}
self.check_interval = check_interval
self.last_check: float = 0
self.running = True
def register_node(self, node_id: str,
health_url: str):
self.healthy_nodes[node_id] = True
def check_node_health(self, node_id: str,
health_url: str) -> bool:
try:
response = requests.get(
health_url, timeout=2
)
return response.status_code == 200
except requests.RequestException:
return False
def get_healthy_nodes(self) -> List[str]:
now = time.time()
if now - self.last_check > self.check_interval:
self._check_all()
self.last_check = now
return [
n for n, h in self.healthy_nodes.items() if h
]
def _check_all(self):
for node_id in list(self.healthy_nodes.keys()):
self.healthy_nodes[node_id] = True
def route_request(self) -> Optional[str]:
healthy = self.get_healthy_nodes()
if not healthy:
return None
return healthy[hash(time.time()) % len(healthy)]
router = HealthRouter()
router.register_node("node-1", "http://10.0.0.1:8080/health")
router.register_node("node-2", "http://10.0.0.2:8080/health")
target = router.route_request()
print(f"Routing to: {target}")
Common Mistakes
Mistake 1: Sticky Sessions Without Shared State
If one node handles auth and another handles API calls, the user gets 401. Use shared Redis for session state.
Mistake 2: Ignoring Split-Brain Scenarios
Network partitions can create multiple leaders. Use a distributed consensus protocol or lease-based system.
Mistake 3: Not Handling Graceful Shutdown
When a node shuts down, in-flight requests fail. Implement connection draining before stopping.
Mistake 4: Manual Cluster Membership
Manual node registration leads to stale entries. Use automatic health checks and expiration.
Mistake 5: Uneven Load Distribution
Round-robin routing without load awareness causes hotspots. Use least-connections or weighted routing.
Practice Questions
- What is the purpose of gateway clustering?
- How does leader election work in a gateway cluster?
- What shared state does a clustered gateway need?
- How do you handle zero-downtime deployments of gateway nodes?
- What happens to in-flight requests when a gateway node fails?
Challenge
Build a gateway cluster manager that supports node registration and discovery via Redis, leader election with TTL-based renewal, health checking with automatic unhealthy node removal, and request routing to healthy nodes.
FAQ
Mini Project
Build a gateway cluster that uses Redis for node discovery and shared state, implements leader election for configuration reload tasks, performs health checks every 10 seconds, and routes requests to healthy nodes using a weighted selection Strategy.
What's Next
Learn about Load Balancing Algorithms for request distribution, or explore Gateway Kubernetes for container Orchestration.
Built by the developers of DodaTech
Doda Browser, DodaZIP & Durga Antivirus Pro