Envoy Proxy as API Gateway — Modern High-Performance Service Proxy

In this tutorial, you will learn about Envoy Proxy as API Gateway. We cover key concepts, practical examples, and best practices to help you master this topic.

Envoy Proxy is a high-performance L7 proxy designed for modern microservices architectures, offering advanced routing, circuit breaking, retries, observability, and dynamic configuration via xDS APIs.

What You'll Learn

• Envoy architecture: listeners, clusters, routes, and filters
• Advanced traffic management: weighted routing, circuit breaking, retries
• Envoy xDS APIs for dynamic configuration

Why It Matters

Envoy was built from the ground up for service mesh and API gateway use cases. It provides first-class support for gRPC, WebSocket, HTTP/2, distributed tracing, and observability features that other gateways add as extensions.

Real-World Use

Durga Antivirus Pro uses Envoy as the sidecar proxy in its Kubernetes service mesh. The same Envoy configuration powers the ingress gateway, handling TLS termination, routing to the correct microservice, circuit breaking for failing pods, and exporting detailed metrics to Prometheus.

flowchart LR
    Client["Client"] --> Envoy["Envoy Ingress Gateway"]
    Envoy --> Route["Dynamic Routes xDS Config"]
    Route --> S1["Service A"]
    Route --> S2["Service B"]
    Route --> S3["Service C"]
    Envoy --> Prometheus["Prometheus"]
    Envoy --> Jaeger["Jaeger Tracing"]
    style Envoy fill:#dbeafe,stroke:#2563eb

Basic Envoy Configuration

static_resources:
  listeners:
  - name: listener_0
    address:
      socket_address: { address: 0.0.0.0, port_value: 8080 }
    filter_chains:
    - filters:
      - name: envoy.filters.network.http_connection_manager
        typed_config:
          "@type": type.googleapis.com/envoy.extensions.filters.network.http_connection_manager.v3.HttpConnectionManager
          stat_prefix: ingress_http
          route_config:
            name: local_route
            virtual_hosts:
            - name: api
              domains: ["*"]
              routes:
              - match: { prefix: "/users" }
                route: { cluster: user_service }
              - match: { prefix: "/orders" }
                route: { cluster: order_service }
          http_filters:
          - name: envoy.filters.http.router

Circuit Breaking in Envoy

clusters:
- name: user_service
  connect_timeout: 0.25s
  type: STRICT_DNS
  lb_policy: ROUND_ROBIN
  circuit_breakers:
    thresholds:
    - priority: DEFAULT
      max_connections: 100
      max_pending_requests: 10
      max_requests: 50
      max_retries: 3
  outlier_detection:
    consecutive_5xx: 5
    interval: 30s
    base_ejection_time: 30s
    max_ejection_percent: 50

This configuration limits connections and detects unhealthy instances. After 5 consecutive 5xx errors, the instance is ejected for 30 seconds.

Envoy with Retries and Timeouts

routes:
- match: { prefix: "/users" }
  route:
    cluster: user_service
    timeout: 3s
    retry_policy:
      retry_on: connect-failure, gateway-error, reset
      num_retries: 3
      retry_host_predicate:
      - name: envoy.retry_host_predicates.previous_hosts

Envoy retries failed requests up to 3 times without routing to the previously attempted host.

Common Mistakes

1. Not Setting Timeouts

Without timeouts, a slow upstream holds connections indefinitely. Always set cluster-level and route-level timeouts.

2. Static Configuration in Production

Envoy shines with dynamic xDS configuration. Hardcoding YAML configs misses service discovery and hot reload benefits.

3. Ignoring Outlier Detection

Without outlier ejection, a single failing instance degrades all traffic. Always configure outlier detection for production.

4. Overly Permissive Circuit Breaker Thresholds

Setting max_connections too high allows cascading failures. Start conservative and tune based on traffic patterns.

5. No Access Logging

Envoy captures detailed access logs. Without them, debugging is difficult. Enable access logs to stdout or a file.

Practice Questions

1. What are the key components of Envoy's architecture?
2. How does Envoy's outlier detection differ from a circuit breaker?
3. What is the xDS protocol and why is it important?
4. How does Envoy support distributed tracing?
5. When should you use Envoy over Nginx as an API gateway?

Answers:

1. Listeners (bind ports), filters (Process traffic), routes (match requests), clusters (backend groups), and endpoints (individual instances).
2. Outlier detection ejects individual unhealthy instances; circuit breaking limits connections to the entire cluster. Both work together.
3. xDS is Envoy's dynamic configuration protocol. Control planes push config changes (routes, clusters, listeners) without restarts.
4. Envoy generates trace spans for each request and propagates trace context via headers, integrating with Jaeger, Zipkin, and Datadog.
5. Choose Envoy for service mesh, gRPC, or when you need dynamic configuration via xDS. Choose Nginx for simpler edge proxy needs.

Challenge: Set up Envoy with Docker Compose as an API gateway routing to two backend services. Configure circuit breakers, outlier detection, retries (3 attempts), and access logging. Test what happens when one backend fails.

FAQ

Mini Project

Deploy Envoy with Docker to route traffic to two backend services. Configure circuit breaking (max 5 connections), outlier detection (3 consecutive failures ejects for 15s), retries (2 attempts), and enable access logging. Test by stopping one backend and observing Envoy behavior.

What's Next

Continue with WebSocket Gateway Support for real-time communication, or explore API Gateway Project to build a complete gateway from scratch.