Canary Deployments with API Gateways — Gradual Traffic Shifting
In this tutorial, you'll learn about Canary Deploy. We cover key concepts, practical examples, and best practices to help you understand and apply this topic effectively.
Canary deployments through the API Gateway gradually shift a percentage of traffic to a new version, allowing you to validate changes before full rollout while minimizing Blast Radius.
What You'll Learn
By the end of this lesson, you will implement traffic splitting at the gateway, configure canary release rules, use metrics to drive promotion decisions, and roll back problematic releases automatically.
Why It Matters
Canary deployments reduce deployment risk by exposing only a subset of users to new versions, catching issues before they affect all users.
Real-World Use
Durga Antivirus Pro routes 5 percent of scan traffic to new versions via the gateway, monitoring error rates and latency before gradually increasing to 100 percent.
Canary Deployment Flow
flowchart LR
Traffic[All Traffic]-->Gateway
Gateway-->Split{Split 95/5}
Split-->|95%|Stable[Stable Version]
Split-->|5%|Canary[Canary Version]
Stable-->Metrics[Metrics Comparison]
Canary-->Metrics
Metrics-->|OK|Increase[Increase Canary %]
Metrics-->|Errors|Rollback[Rollback to Stable]
Traffic Split Router
Route traffic between stable and canary versions based on configurable rules.
import random
from typing import Dict, Optional, Tuple, Callable
import hashlib
class CanaryRouter:
def __init__(self):
self.routes: Dict[str, Dict] = {}
def add_canary(self, path: str,
stable_backend: str,
canary_backend: str,
canary_percent: float = 5.0,
sticky_key: Optional[str] = None):
self.routes[path] = {
"stable": stable_backend,
"canary": canary_backend,
"percent": canary_percent,
"sticky_key": sticky_key,
}
def route(self, path: str,
client_id: Optional[str] = None
) -> Tuple[str, str]:
config = self.routes.get(path)
if not config:
return "stable", "default-backend"
use_canary = False
if config["sticky_key"] and client_id:
hash_val = int(
hashlib.md5(
client_id.encode()
).hexdigest(),
16
)
use_canary = (hash_val % 100) < config["percent"]
else:
use_canary = random.random() * 100 < config["percent"]
if use_canary:
return "canary", config["canary"]
return "stable", config["stable"]
def update_canary_percent(self, path: str,
percent: float):
if path in self.routes:
self.routes[path]["percent"] = min(100.0, max(0.0, percent))
router = CanaryRouter()
router.add_canary("/api/scan", "scan-v1:8080",
"scan-v2:8080", canary_percent=10.0,
sticky_key="user_id")
for i in range(10):
version, backend = router.route(
"/api/scan", f"user-{i}"
)
print(f"user-{i}: {version} -> {backend}")
Metrics-Driven Promotion
Automatically promote canary based on error rate and latency metrics.
from typing import Dict, Optional, Tuple
import time
class CanaryPromoter:
def __init__(self):
self.canaries: Dict[str, Dict] = {}
self.thresholds = {
"error_rate": 1.0,
"latency_p95_ms": 500,
}
def register_canary(self, path: str,
initial_percent: float = 5.0,
promotion_steps: list = None):
self.canaries[path] = {
"percent": initial_percent,
"steps": promotion_steps or [5, 10, 25, 50, 100],
"current_step": 0,
"started": time.time(),
"errors": 0,
"total": 0,
"latencies": [],
}
def record_result(self, path: str,
is_error: bool,
latency_ms: float):
canary = self.canaries.get(path)
if not canary:
return
canary["total"] += 1
if is_error:
canary["errors"] += 1
canary["latencies"].append(latency_ms)
def get_error_rate(self, path: str) -> float:
canary = self.canaries.get(path)
if not canary or canary["total"] == 0:
return 0.0
return canary["errors"] / canary["total"] * 100
def should_promote(self, path: str) -> Tuple[bool, float]:
canary = self.canaries.get(path)
if not canary:
return False, 0.0
error_rate = self.get_error_rate(path)
if error_rate > self.thresholds["error_rate"]:
return False, error_rate
if canary["current_step"] >= len(canary["steps"]) - 1:
return False, error_rate
elapsed = time.time() - canary["started"]
if elapsed < 300:
return False, error_rate
next_percent = canary["steps"][canary["current_step"] + 1]
return True, next_percent
def promote(self, path: str) -> Optional[float]:
should, next_val = self.should_promote(path)
if should:
canary = self.canaries[path]
canary["current_step"] += 1
new_percent = canary["steps"][canary["current_step"]]
canary["percent"] = new_percent
canary["errors"] = 0
canary["total"] = 0
canary["latencies"] = []
canary["started"] = time.time()
return new_percent
return None
promoter = CanaryPromoter()
promoter.register_canary("/api/scan")
for _ in range(100):
promoter.record_result("/api/scan", False, 50)
should, pct = promoter.should_promote("/api/scan")
print(f"Should promote: {should}, to: {pct}%")
A/B Testing Through Canary Routing
Use canary routing for A/B testing different versions.
from typing import Dict, Optional, Any
import hashlib
class ABTestRouter:
def __init__(self):
self.experiments: Dict[str, Dict] = {}
def add_experiment(self, name: str,
variants: Dict[str, float],
path: str):
self.experiments[name] = {
"variants": variants,
"path": path,
}
def get_variant(self, experiment: str,
user_id: str) -> Optional[str]:
config = self.experiments.get(experiment)
if not config:
return None
hash_val = int(
hashlib.md5(
f"{experiment}:{user_id}".encode()
).hexdigest(),
16
)
total = sum(config["variants"].values())
bucket = hash_val % total
cumulative = 0
for variant, weight in config["variants"].items():
cumulative += weight
if bucket < cumulative:
return variant
return None
def route_for_experiment(self, experiment: str,
user_id: str,
path: str) -> Optional[str]:
variant = self.get_variant(experiment, user_id)
if variant:
return f"{path}/{variant}"
return path
ab = ABTestRouter()
ab.add_experiment(
"scan-ui",
{"control": 50, "variant-a": 25, "variant-b": 25},
"/api/scan/ui"
)
for uid in ["user-1", "user-2", "user-3", "user-4"]:
route = ab.route_for_experiment("scan-ui", uid, "/api/scan/ui")
print(f"{uid}: {route}")
Common Mistakes
Mistake 1: Not Using Sticky Sessions
Without stickiness, the same user bounces between stable and canary, experiencing inconsistent behavior.
Mistake 2: Insufficient Canary Traffic
Less than 1 percent traffic may not trigger issues that appear under higher load.
Mistake 3: Manual Rollback
Automated rollback based on metrics is essential. Manual rollback is too slow during incidents.
Mistake 4: Ignoring Database Schema Changes
Canary code that writes to a modified schema breaks stable code reading the same data.
Mistake 5: No Metrics Comparison Baseline
Compare canary metrics against stable metrics, not absolute thresholds, to account for normal fluctuations.
Practice Questions
- What is the difference between canary and blue-green deployment?
- How do you ensure a user consistently sees the same version?
- What metrics should trigger a canary rollback?
- How do you handle database schema changes during canary deployments?
- What is the minimum canary percentage for meaningful validation?
Challenge
Build a canary deployment system for the gateway that supports percentage-based traffic splitting with user stickiness, metrics-driven automatic promotion through 5-10-25-50-100 percent stages, and automatic rollback if error rates exceed 2 percent.
FAQ
Mini Project
Build a canary deployment system for the gateway that routes traffic based on percentage with user stickiness, supports multiple simultaneous canaries on different endpoints, automatically promotes through configurable steps based on error rate and latency metrics, and rolls back instantly if thresholds are breached.
What's Next
Learn about Blue-Green Deployments for zero-downtime releases, or explore Load Balancing Algorithms for traffic distribution strategies.
Built by the developers of DodaTech
Doda Browser, DodaZIP & Durga Antivirus Pro