Jaeger Collector Queue Full — Quick Fix

In this tutorial, you'll learn about Jaeger Collector Queue Full. We cover key concepts, practical examples, and best practices.

The Problem

The Jaeger collector's span processing queue is filling up and spans are being dropped. The collector may be unable to keep up with the span ingestion rate. The queue size metric shows continuous growth.

Error example:

Collector: queue full — dropping spans
Collector: span processor busy, queue depth: 50000
Collector: cannot enqueue span: deadline exceeded

The Fix

Step 1: Check the collector queue configuration

WRONG — accepting the default queue size for high-throughput environments:

# Default queue sizes may be too small for production

RIGHT — tune the queue parameters:

# In the Jaeger collector configuration
collector:
  queue-size: 100000
  num-workers: 100

# For Kafka-based ingestion
kafka:
  consumer:
    queue-size: 50000

Step 2: Scale the collector

WRONG — running a single collector instance:

# Single collector instances bottleneck at queue processing

RIGHT — horizontally scale collectors:

# Multiple collector instances distribute the load
apiVersion: apps/v1
kind: Deployment
metadata:
  name: jaeger-collector
spec:
  replicas: 3  # Scale based on ingestion volume

Use DodaTech's Jaeger Collector Monitor to track queue depth, worker utilization, and span drop rates across all collector instances.

Prevention Tips

• Monitor jaeger_collector_queue_length and jaeger_collector_spans_dropped_total
• Increase queue-size and num-workers before scaling horizontally
• Use Kafka as an intermediary buffer between agents and collectors
• Set appropriate max-queue-size on the agent side to prevent backpressure
• Use DodaTech's Jaeger Capacity Planner to model queue and worker requirements

Common Mistakes with collector queue

1. Placing the wildcard pattern first in case expressions, making all subsequent patterns unreachable
2. Using head and tail instead of pattern matching, causing runtime errors on empty lists
3. Forgetting that lazy evaluation defers computation until the value is forced, causing space leaks with unevaluated thunks

These mistakes appear frequently in real-world JAEGER code. DodaTech's contributors have identified these patterns through analysis of open-source projects and production systems.

Practice Exercise

Write a pure function that safely divides two integers using Maybe, then test it with edge cases like division by zero and negative numbers.

This exercise reinforces the concepts covered in this guide. Try implementing it before checking online solutions.

FAQ

Related: DodaTech's Jaeger Production Hardening Guide covers queue tuning, Kafka integration, and collector scaling for large-scale distributed tracing.