Linkerd Retry Budget Exhausted Immediately

You encounter a linkerd configuration issue that prevents your workflow from completing. This guide walks through the fix step by step.

Wrong ❌

apiVersion: linkerd.io/v1alpha2
kind: ServiceProfile
metadata: { name: my-service.default.svc.cluster.local }
spec:
  routes:
    - name: GET /api
      condition: { method: GET, pathRegex: /api/.* }
      isRetryable: true
      # Missing retryBudget

Wrong Output

Retries fail immediately after 2 retries. Default budget too small.

Right ✅

apiVersion: linkerd.io/v1alpha2
kind: ServiceProfile
metadata: { name: my-service.default.svc.cluster.local }
spec:
  routes:
    - name: GET /api
      condition: { method: GET, pathRegex: /api/.* }
      isRetryable: true
  retryBudget:
    minRetriesPerSecond: 10
    retryRatio: 0.2
    ttl: 10s

Right Output

Retries within budget. 503 failures trigger retries. Retry budget: 10/s, 20% ratio.

Prevention

• Configure retryBudget with appropriate minRetriesPerSecond.
• Set retryRatio to 0.2 as a starting point.
• Use TTL of 10s for retry budget window.
• Mark only idempotent routes as isRetryable.
• Test with fault injection before production.

DodaTech applies similar defensive patterns across Doda Browser, DodaZIP, and Durga Antivirus Pro infrastructure for production reliability.

Common Mistakes with retry budget

1. Using foldl instead of foldl' causing stack overflow on large lists
2. Forgetting deriving (Show, Eq) on custom data types needed for debugging
3. Placing the wildcard pattern first in case expressions, making all subsequent patterns unreachable

These mistakes appear frequently in real-world LINKERD code. DodaTech's contributors have identified these patterns through analysis of open-source projects and production systems.

Practice Exercise

Write a pure function that safely divides two integers using Maybe, then test it with edge cases like division by zero and negative numbers.

This exercise reinforces the concepts covered in this guide. Try implementing it before checking online solutions.

FAQ

This quick fix is part of the DodaTech infrastructure engineering series. Learn more at DodaTech tutorials.