Skip to content

OpenTelemetry Batch Span Processor Not Flushing Fix

DodaTech Updated 2026-06-24 3 min read

In this tutorial, you'll learn about OpenTelemetry Batch Span Processor Not Flushing Fix. We cover key concepts, practical examples, and best practices.

Your OpenTelemetry application loses the last few spans on shutdown — the tracing backend shows fewer spans than expected, missing the final request traces. The batch span processor does not flush before the process exits.

The Problem

from opentelemetry.sdk.trace import TracerProvider
from opentelemetry.sdk.trace.export import BatchSpanProcessor
from opentelemetry.exporter.otlp.proto.grpc.trace_exporter import OTLPSpanExporter

provider = TracerProvider()
processor = BatchSpanProcessor(OTLPSpanExporter())
provider.add_span_processor(processor)
trace.set_tracer_provider(provider)

# Application runs...
# Application exits without flushing

The last few spans are still in the batch buffer when the process exits. The exporter never sends them.

Step-by-Step Fix

1. Force flush on shutdown

import atexit
from opentelemetry import trace
from opentelemetry.sdk.trace import TracerProvider
from opentelemetry.sdk.trace.export import BatchSpanProcessor
from opentelemetry.exporter.otlp.proto.grpc.trace_exporter import OTLPSpanExporter

provider = TracerProvider()
processor = BatchSpanProcessor(OTLPSpanExporter())
provider.add_span_processor(processor)
trace.set_tracer_provider(provider)

# Flush all remaining spans on exit
atexit.register(processor.shutdown)

2. Configure batch parameters

from opentelemetry.sdk.trace.export import BatchSpanProcessor

processor = BatchSpanProcessor(
    OTLPSpanExporter(),
    max_queue_size=2048,        # Max spans in queue
    max_export_batch_size=512,  # Max spans per batch
    schedule_delay_millis=5000, # Export every 5 seconds
    export_timeout_millis=30000 # Timeout per export
)

3. Use SimpleSpanProcessor for low-throughput apps

from opentelemetry.sdk.trace.export import SimpleSpanProcessor

# Exports spans immediately — no buffering
processor = SimpleSpanProcessor(OTLPSpanExporter())
provider.add_span_processor(processor)

Warning: SimpleSpanProcessor exports synchronously, which adds latency to each span. Use only for development or very low throughput.

4. Handle graceful shutdown in web frameworks

from flask import Flask
from opentelemetry import trace
from opentelemetry.sdk.trace import TracerProvider
from opentelemetry.sdk.trace.export import BatchSpanProcessor
from opentelemetry.exporter.otlp.proto.grpc.trace_exporter import OTLPSpanExporter

app = Flask(__name__)

provider = TracerProvider()
processor = BatchSpanProcessor(OTLPSpanExporter())
provider.add_span_processor(processor)
trace.set_tracer_provider(provider)

@app.teardown_appcontext
def shutdown(exception=None):
    processor.force_flush()

# Also register for process exit
import atexit
atexit.register(lambda: processor.shutdown())

5. Monitor processor metrics

from opentelemetry.sdk.trace.export import BatchSpanProcessor

processor = BatchSpanProcessor(OTLPSpanExporter())

# After test run
print(f"Queue size: {processor.queue.qsize()}")
print(f"Dropped spans: {processor.dropped_spans}")
processor.force_flush()
print("Flushed remaining spans")

Expected output:

Queue size: 0
Dropped spans: 0
Flushed remaining spans

Prevention Tips

  • Register processor.shutdown() with atexit for graceful shutdown
  • Configure schedule_delay_millis to match your latency requirements
  • Use force_flush() before long idle periods
  • Monitor queue size to detect export bottlenecks
  • Use BatchSpanProcessor for production, SimpleSpanProcessor for development

Common Mistakes with batch processor

  1. Forgetting deriving (Show, Eq) on custom data types needed for debugging
  2. Placing the wildcard pattern first in case expressions, making all subsequent patterns unreachable
  3. Using head and tail instead of pattern matching, causing runtime errors on empty lists

These mistakes appear frequently in real-world OPENTELEMETRY code. DodaTech's contributors have identified these patterns through analysis of open-source projects and production systems.

Practice Exercise

Write a pure function that safely divides two integers using Maybe, then test it with edge cases like division by zero and negative numbers.

This exercise reinforces the concepts covered in this guide. Try implementing it before checking online solutions.

FAQ

### What happens when the batch queue is full?

When max_queue_size is exceeded, the oldest spans are dropped to make room for new ones. Increase max_queue_size for high-throughput applications, or check the exporter endpoint for connectivity issues causing queue buildup.

What's the difference between force_flush and shutdown?

force_flush() exports all queued spans without stopping the processor — the processor continues accepting new spans. shutdown() exports remaining spans and stops the processor — no new spans can be added after shutdown.

How do I handle span loss in high-throughput scenarios?

(1) Increase max_queue_size to 4096+, (2) reduce schedule_delay_millis to 1-2 seconds, (3) increase max_export_batch_size to 1024+, (4) use multiple processors for parallel export, (5) add a SimpleSpanProcessor as a fallback for critical spans.

Built by the developers of DodaTech

Doda Browser, DodaZIP & Durga Antivirus Pro