Choreography Saga — Decentralized Distributed Transaction Coordination
In this tutorial, you will learn about Choreography Saga. We cover key concepts, practical examples, and best practices to help you master this topic.
Choreography saga coordinates distributed transactions across Microservices using events emitted by each service to trigger the next step, with compensating events to undo failures without a central coordinator.
What You'll Learn
By the end of this lesson you will understand how choreography sagas work, implement an event-driven saga for order processing, design compensating transactions for rollback, handle failure scenarios, and compare choreography with Orchestration.
Why It Matters
In a monolith, a database Transaction ensures all-or-nothing execution. In microservices, each service has its own database. Distributed transactions via two-phase commit are impractical. Sagas provide a practical alternative by breaking a transaction into steps with compensating rollbacks.
Real-World Use
DodaZIP's subscription upgrade flow uses a choreography saga. When a user upgrades, the billing service emits SubscriptionUpgraded, the license service provisions new features, the notification service sends confirmation, and the analytics service tracks the event. If any step fails, compensating events undo previous steps.
flowchart LR
A[Order Service] -->|OrderPlaced| B[Payment Service]
B -->|PaymentReceived| C[Inventory Service]
C -->|StockConfirmed| D[Shipping Service]
B -->|PaymentFailed| A
C -->|OutOfStock| B
D -->|Shipped| E[Complete]
style A fill:#2d3748,color:#fff
style B fill:#2d3748,color:#fff
style C fill:#2d3748,color:#fff
style D fill:#2d3748,color:#fff
Choreography Saga Principles
How choreography sagas work.
# choreography_principles.py
# Choreography saga fundamentals
def choreography_principles():
print("Choreography Saga Principles")
print("=" * 40)
print()
principles = [
{
"principle": "Event-Driven Coordination",
"desc": "Each service emits events that trigger the next service. No central coordinator.",
"example": "OrderPlaced -> Payment Service starts processing"
},
{
"principle": "Local Transactions Only",
"desc": "Each service updates its own database independently within a local ACID transaction.",
"example": "Order service saves order + publishes OrderPlaced in one DB transaction"
},
{
"principle": "Compensating Events",
"desc": "If a step fails, previous steps emit compensating events to undo their work.",
"example": "PaymentFailed event triggers OrderCancelled in order service"
},
{
"principle": "Eventual Consistency",
"desc": "The system becomes consistent over time as events propagate. There is a window of inconsistency.",
"example": "Order shows 'confirmed' for 100ms before inventory confirms stock"
},
{
"principle": "Idempotent Handlers",
"desc": "Event handlers must handle duplicate events safely.",
"example": "Processing PaymentReceived twice should not charge the customer twice"
},
]
for p in principles:
print(f"{p['principle']:30s}")
print(f" {p['desc']}")
print(f" Example: {p['example']}")
print()
choreography_principles()
Saga Implementation
Order processing choreography saga.
# saga_implementation.py
# Choreography saga implementation
def saga_implementation():
print("Choreography Saga - Order Processing")
print("=" * 45)
print()
saga_code = """
import json
class OrderService:
"""Step 1: Place the order."""
def place_order(self, order_data):
with self.db.transaction():
order = self.save_order(order_data)
self.event_bus.publish("order.placed", {
"order_id": order.id,
"customer_id": order.customer_id,
"total": order.total,
"items": order.items
})
return order
def handle_payment_failed(self, event):
"""Compensating action: cancel the order."""
with self.db.transaction():
self.cancel_order(event["data"]["order_id"])
self.event_bus.publish("order.cancelled", event["data"])
class PaymentService:
"""Step 2: Process payment."""
def handle_order_placed(self, event):
data = event["data"]
try:
payment = self.process_payment(
data["customer_id"], data["total"]
)
with self.db.transaction():
self.save_payment(payment)
self.event_bus.publish("payment.received", {
"order_id": data["order_id"],
"payment_id": payment.id,
"amount": data["total"]
})
except PaymentError as e:
with self.db.transaction():
self.event_bus.publish("payment.failed", {
"order_id": data["order_id"],
"reason": str(e)
})
class InventoryService:
"""Step 3: Reserve inventory."""
def handle_payment_received(self, event):
data = event["data"]
try:
for item in data.get("items", []):
self.reserve_stock(item["product_id"], item["quantity"])
with self.db.transaction():
self.event_bus.publish("inventory.reserved", data)
except OutOfStockError as e:
with self.db.transaction():
self.event_bus.publish("inventory.failed", {
**data, "reason": str(e)
})
def handle_order_cancelled(self, event):
"""Compensating action: release reserved stock."""
for item in event["data"].get("items", []):
self.release_stock(item["product_id"], item["quantity"])
"""
print(saga_code)
saga_implementation()
Failure Scenarios
Handling various failure modes in a saga.
# failure_scenarios.py
# Saga failure handling
def failure_scenarios():
print("Saga Failure Scenarios and Compensation")
print("=" * 45)
print()
scenarios = [
{
"step": "Order Placed",
"failure": "Payment declined",
"compensation": "Order cancelled (automatic)",
"description": "Payment service emits payment.failed. Order service cancels the order."
},
{
"step": "Payment Received",
"failure": "Out of stock",
"compensation": "Refund payment + cancel order",
"description": "Inventory service emits inventory.failed. Payment service refunds. Order service cancels."
},
{
"step": "Inventory Reserved",
"failure": "Shipping unavailable",
"compensation": "Release inventory + refund + cancel",
"description": "Shipping service unavailable. Inventory releases stock. Payment refunds. Order cancels."
},
{
"step": "Shipped",
"failure": "Delivery failed",
"compensation": "Return to sender + refund",
"description": "After shipping, delivery fails. Return process initiated, refund issued."
},
]
for s in scenarios:
print(f"Step: {s['step']}")
print(f"Failure: {s['failure']}")
print(f"Rollback: {s['compensation']}")
print(f"Details: {s['description']}")
print()
failure_scenarios()
Choreography vs Orchestration
Comparing both saga coordination approaches.
# vs_orchestration.py
# Choreography vs orchestration
def compare_saga_approaches():
print("Choreography vs Orchestration Saga")
print("=" * 40)
print()
comparisons = [
("Coordination", "Event-driven, decentralized", "Command-driven, central coordinator"),
("Coupling", "Loose (services know events only)", "Tighter (services know orchestrator)"),
("Complexity", "Higher (tracking event flow)", "Lower (single flow definition)"),
("Visibility", "Harder to trace flow", "Easy to see all steps"),
("Single Point of Failure", "No central point", "Orchestrator is SPOF"),
("Best For", "Simple chains, few services", "Complex flows, many services"),
("Testing", "Harder (distributed)", "Easier (orchestrator testable)"),
]
print(f"{'Aspect':25s} {'Choreography':35s} {'Orchestration':35s}")
print("-" * 95)
for aspect, choreo, orchs in comparisons:
print(f"{aspect:25s} {choreo:35s} {orchs:35s}")
compare_saga_approaches()
Common Mistakes
Not implementing compensating events: Every step that makes a change must have a compensating event. Missing compensation leaves the system in an inconsistent state.
Non-idempotent event handlers: Network issues cause duplicate event delivery. Without idempotency, the same event is processed multiple times, causing duplicate payments or inventory deductions.
Missing timeout handling: A service may never emit a success or failure event. Implement timeouts and timeout handlers as compensations.
Circular sagas: Service A emits an event that triggers Service B, which emits an event back to Service A, creating an infinite loop. Use event versioning or trace IDs to prevent cycles.
Ignoring partial failures: If the compensating action also fails, the system enters an inconsistent state. Implement manual intervention or retry queues with exponential backoff.
Practice Questions
What is a saga in microservices? A sequence of local transactions where each step publishes an event that triggers the next step, with compensating transactions for rollback.
How does choreography saga differ from distributed transactions? Sagas use eventual consistency with compensating actions. Distributed transactions use two-phase commit with immediate consistency.
What is a compensating transaction? An action that undoes the effects of a previous step in the saga, such as cancelling an order or issuing a refund.
Why must saga event handlers be idempotent? Network retries can deliver the same event multiple times. Idempotent handlers produce the same result regardless of how many times the event is processed.
Challenge: Design a choreography saga for a hotel booking system. Steps: reserve room, charge card, send confirmation. Failure scenarios: card declined, room no longer available, confirmation delivery failure. Define all events and compensating actions.
FAQ
Mini Project
Design a choreography saga for a SaaS subscription management system. Steps: create subscription, provision resources, send welcome email, update billing cycle. Define compensating events for each step. Handle partial provisioning failure and billing failure scenarios.
def subscription_saga():
print("SaaS Subscription - Choreography Saga")
print("=" * 45)
print()
print("Events Flow:")
print()
print(" 1. Billing Service")
print(" Event: subscription.created")
print(" Comp: subscription.cancelled (refund)")
print()
print(" 2. Provisioning Service")
print(" Event: resources.provisioned")
print(" Comp: resources.deprovisioned")
print()
print(" 3. Notification Service")
print(" Event: welcome.sent")
print(" Comp: N/A (no side effect)")
print()
print(" 4. Analytics Service")
print(" Event: subscription.tracked")
print(" Comp: N/A (logging only)")
print()
print("Failure: Provisioning fails after billing")
print(" -> Provisioning emits provisioning.failed")
print(" -> Billing handles: subscription.cancelled (refund)")
print(" -> Notification handles: welcome.cancelled")
subscription_saga()
What's Next
Next: Orchestration Saga for centrally coordinated sagas.
Built by the developers of DodaTech
Doda Browser, DodaZIP & Durga Antivirus Pro