Logging and Monitoring — Complete Gateway Observability Guide

In this tutorial, you will learn about Logging and Monitoring. We cover key concepts, practical examples, and best practices to help you master this topic.

The API Gateway is the ideal location for centralized logging and monitoring. Since all traffic passes through the gateway, it provides a complete view of API usage, performance, and errors.

What You'll Learn

You'll learn what to log at the gateway, key metrics to monitor, and how to integrate with Observability platforms.

Why It Matters

Monitoring at the gateway captures 100% of API traffic. It provides the earliest warning of issues and a single source of truth for API analytics.

Real-World Use

An API gateway logs every request with a unique correlation ID. When a user reports a slow response, the operations team traces the correlation ID through the gateway logs to identify which backend service caused the delay.

Implementation

# Kong logging plugin with OpenTelemetry
plugins:
  - name: opentelemetry
    config:
      endpoint: "http://otel-collector:4318/v1/traces"
      headers:
        - "Authorization: Bearer your-token"
      resource_attributes:
        - "service.name:api-gateway"
        - "deployment.environment:production"

  - name: http-log
    config:
      http_endpoint: "http://log-aggregator:9200/_bulk"
      method: POST
      headers:
        - "Content-Type: application/json"
      timeout: 10000
      keepalive: 60000
      retry_count: 5

# NGINX logging with JSON format
log_format json_combined '{'
    '"time":"$time_iso8601",'
    '"request_id":"$request_id",'
    '"remote_addr":"$remote_addr",'
    '"request":"$request",'
    '"status":$status,'
    '"body_bytes":$body_bytes_sent,'
    '"request_time":$request_time,'
    '"upstream_addr":"$upstream_addr",'
    '"upstream_status":$upstream_status,'
    '"http_referer":"$http_referer",'
    '"http_user_agent":"$http_user_agent"'
'}';

server {
    access_log /var/log/nginx/api-access.log json_combined;
}

Common Mistakes

| Mistake | Fix | |---------|-----| | Not logging correlation IDs | Cannot trace requests across services | Add unique request ID on every request | | Over-logging sensitive data | PII and tokens in logs | Scrub sensitive fields before logging | | No log retention policy | Logs fill disk | Set retention: 90 days hot, 1 year cold | | Not monitoring gateway metrics | Blind to performance issues | Track latency, error rates, throughput | | No alerting on error spikes | Issues discovered by users | Alert on 5xx rate exceeding threshold |

What's Next

Learn about Kong API Gateway in depth.