Rate Limit Monitoring — Tracking API Throttling with Prometheus and Grafana

In this tutorial, you will learn about Rate Limit Monitoring. We cover key concepts, practical examples, and best practices to help you master this topic.

Rate limit monitoring uses Prometheus metrics to track how often limits are hit, which endpoints trigger the most throttling, and whether current limits are correctly sized for actual traffic patterns.

What You'll Learn

• What rate limit metrics to collect in production
• How to expose Prometheus metrics from your rate limiter
• How to build a Grafana dashboard for rate limit monitoring

Why It Matters

Without monitoring, you cannot know if your rate limits are too restrictive (blocking legitimate users) or too permissive (allowing abuse). Metrics-driven rate limit tuning ensures the right balance between protection and Accessibility.

Real-World Use

DodaTech's rate limiter exposes Prometheus counters for total requests, limited requests, and requests by endpoint. The operations team has a Grafana dashboard that shows real-time throttle rates, top-limited clients, and trend lines for capacity planning.

flowchart LR
    A["API Server"] --> B["Rate Limiter"]
    B --> C["Prometheus Metrics\nrate_limit_total\nrate_limit_blocked\nrate_limit_by_endpoint"]
    C --> D["Prometheus\nServer"]
    D --> E["Grafana\nDashboard"]
    E --> F["Alert Manager\nNotify on high throttle rate"]
    style B fill:#dbeafe,stroke:#2563eb
    style E fill:#fef3c7,stroke:#d97706
    style F fill:#fecaca,stroke:#dc2626

Prometheus Metrics

from prometheus_client import Counter, Histogram, Gauge, generate_latest
from flask import Flask, Response
import time

app = Flask(__name__)

# Rate limit metrics
rate_limit_total = Counter(
    'rate_limit_total',
    'Total requests processed by rate limiter',
    ['endpoint', 'method', 'tier']
)

rate_limit_blocked = Counter(
    'rate_limit_blocked',
    'Requests blocked by rate limiter',
    ['endpoint', 'method', 'reason']
)

rate_limit_active_keys = Gauge(
    'rate_limit_active_keys',
    'Number of API keys currently being rate limited'
)

rate_limit_latency = Histogram(
    'rate_limit_check_duration_seconds',
    'Time spent checking rate limits',
    buckets=[0.001, 0.005, 0.01, 0.025, 0.05, 0.1]
)

@app.route('/metrics')
def metrics():
    return Response(generate_latest(), mimetype='text/plain')

Integrating Metrics with Rate Limiter

def rate_limit_middleware():
    def decorator(f):
        @wraps(f)
        def decorated(*args, **kwargs):
            api_key = extract_api_key()
            endpoint = request.path
            method = request.method

            start_time = time.time()

            # Check rate limit
            allowed, info = check_rate_limit(api_key)

            # Record latency
            rate_limit_latency.observe(time.time() - start_time)

            # Record metrics
            tier = info.get('tier', 'unknown')
            rate_limit_total.labels(
                endpoint=endpoint,
                method=method,
                tier=tier
            ).inc()

            if not allowed:
                rate_limit_blocked.labels(
                    endpoint=endpoint,
                    method=method,
                    reason=info.get('reason', 'limit_exceeded')
                ).inc()
                return jsonify({"error": "rate_limited"}), 429

            return f(*args, **kwargs)
        return decorated
    return decorator

Grafana Dashboard Queries

-- Rate of blocked requests per minute
sum(rate(rate_limit_blocked_total[5m])) by (reason)

-- Throttle rate (percentage of requests blocked)
sum(rate(rate_limit_blocked_total[5m])) /
sum(rate(rate_limit_total[5m])) * 100

-- Top limited endpoints
topk(10, sum(rate(rate_limit_blocked_total[24h])) by (endpoint))

-- Active API keys with exhausted limits
rate_limit_active_keys

-- Rate limit check latency (p95)
histogram_quantile(0.95,
  sum(rate(rate_limit_check_duration_seconds_bucket[5m])) by (le)
)

Alerting Rules

# prometheus-rules.yml
groups:
  - name: rate_limiting_alerts
    rules:
      - alert: HighThrottleRate
        expr: |
          sum(rate(rate_limit_blocked_total[5m])) /
          sum(rate(rate_limit_total[5m])) * 100 > 10
        for: 5m
        labels:
          severity: warning
        annotations:
          summary: "Throttle rate above 10%"

      - alert: RateLimitLatencyHigh
        expr: |
          histogram_quantile(0.99,
            rate(rate_limit_check_duration_seconds_bucket[5m])
          ) > 0.1
        for: 5m
        labels:
          severity: critical
        annotations:
          summary: "Rate limit check latency above 100ms"

      - alert: AllKeysExhausted
        expr: rate_limit_active_keys > 1000
        for: 5m
        labels:
          severity: info
        annotations:
          summary: "Over 1000 API keys currently rate limited"

Common Mistakes

1. Not Collecting Per-Endpoint Metrics

Aggregate metrics hide which endpoints are problematic. Track per-endpoint to identify abuse targets.

2. Ignoring Latency of Rate Limit Checks

Rate limit checks add latency. Monitor it to ensure Redis calls or database queries are not slowing your API.

3. Not Setting Up Alerts

Rate limit abuse often escalates quietly. Set alerts for sudden spikes in blocked requests.

4. Only Monitoring Blocked Requests

Track both allowed and blocked requests. A sudden drop in allowed requests may indicate a configuration error.

5. Not Keeping Historical Data

Rate limit trends over weeks and months reveal usage patterns and help calibrate limits. Retain Prometheus data.

Practice Questions

1. What Prometheus metrics should a rate limiter expose?
2. How do you calculate throttle rate percentage?
3. What latency threshold should trigger an alert?
4. Why track per-endpoint rate limit metrics?
5. What retention period is recommended for rate limit metrics?

Answers

1. Total requests, blocked requests, active keys, and latency histograms. 2. sum(rate(blocked[5m])) / sum(rate(total[5m])) * 100. 3. P99 latency above 100ms. 4. To identify which endpoints are being attacked or have inappropriate limits. 5. At least 90 days for trend analysis.

Challenge

Build a complete rate limit monitoring stack that: instruments a Flask app with Prometheus rate limit metrics, creates a docker-compose setup with Prometheus and Grafana, provides pre-built dashboards for throttle rate and latency, and includes alert rules for common rate limit anomalies.

FAQ

Mini Project

Create a docker-compose-based monitoring stack with: a rate-limited Flask API instrumented with Prometheus, Prometheus server for metrics collection, Grafana with a pre-built rate limit dashboard showing throttle rate, top endpoints, active limited keys, and latency heatmaps, plus Alertmanager for threshold alerts.

What's Next

• Learn about rate limit alerting for proactive Incident Response
• Explore rate limit bypass prevention techniques
• Continue to algorithm deep dives for implementation