Apache APISIX Upstream Marked Unhealthy

You encounter a apisix configuration issue that prevents your workflow from completing. This guide walks through the fix step by step.

Wrong ❌

{
  "upstream": {
    "type": "roundrobin",
    "nodes": { "backend:3000": 1 },
    "checks": {
      "active": {
        "type": "http",
        "http_path": "/health",
        "healthy": { "interval": 2, "successes": 1 },
        "unhealthy": { "interval": 1, "http_failures": 1 }
      }
    }
  }
}

Wrong Output

Upstream nodes marked unhealthy immediately. Single failure triggers unhealthy with 1s recheck interval.

Right ✅

{
  "upstream": {
    "type": "roundrobin",
    "nodes": { "backend:3000": 1, "backend2:3000": 1 },
    "checks": {
      "active": {
        "type": "http",
        "http_path": "/health",
        "timeout": 5,
        "concurrency": 1,
        "healthy": { "interval": 10, "successes": 3 },
        "unhealthy": { "interval": 5, "http_failures": 5, "timeouts": 3 }
      },
      "passive": {
        "http_failures": 10,
        "timeouts": 7
      }
    }
  }
}

Right Output

Nodes require 3 successes to become healthy, 5 failures to become unhealthy. Passive checks add extra tolerance.

Prevention

• Set realistic healthy/unhealthy thresholds.
• Use both active and passive checks.
• Passive check failures should be higher than active.
• Use multiple upstream nodes for fault tolerance.
• Monitor upstream health via APISIX dashboard.

DodaTech applies similar defensive patterns across Doda Browser, DodaZIP, and Durga Antivirus Pro infrastructure for production reliability.

Common Mistakes with upstream health

1. Misunderstanding that String is [Char] with poor performance for large text operations
2. Using foldl instead of foldl' causing stack overflow on large lists
3. Forgetting deriving (Show, Eq) on custom data types needed for debugging

These mistakes appear frequently in real-world APISIX code. DodaTech's contributors have identified these patterns through analysis of open-source projects and production systems.

Practice Exercise

Write a pure function that safely divides two integers using Maybe, then test it with edge cases like division by zero and negative numbers.

This exercise reinforces the concepts covered in this guide. Try implementing it before checking online solutions.

FAQ

This quick fix is part of the DodaTech infrastructure engineering series. Learn more at DodaTech tutorials.