OpenTelemetry Batch Span Processor Not Flushing Fix
In this tutorial, you'll learn about OpenTelemetry Batch Span Processor Not Flushing Fix. We cover key concepts, practical examples, and best practices.
Your OpenTelemetry application loses the last few spans on shutdown — the tracing backend shows fewer spans than expected, missing the final request traces. The batch span processor does not flush before the process exits.
The Problem
from opentelemetry.sdk.trace import TracerProvider
from opentelemetry.sdk.trace.export import BatchSpanProcessor
from opentelemetry.exporter.otlp.proto.grpc.trace_exporter import OTLPSpanExporter
provider = TracerProvider()
processor = BatchSpanProcessor(OTLPSpanExporter())
provider.add_span_processor(processor)
trace.set_tracer_provider(provider)
# Application runs...
# Application exits without flushing
The last few spans are still in the batch buffer when the process exits. The exporter never sends them.
Step-by-Step Fix
1. Force flush on shutdown
import atexit
from opentelemetry import trace
from opentelemetry.sdk.trace import TracerProvider
from opentelemetry.sdk.trace.export import BatchSpanProcessor
from opentelemetry.exporter.otlp.proto.grpc.trace_exporter import OTLPSpanExporter
provider = TracerProvider()
processor = BatchSpanProcessor(OTLPSpanExporter())
provider.add_span_processor(processor)
trace.set_tracer_provider(provider)
# Flush all remaining spans on exit
atexit.register(processor.shutdown)
2. Configure batch parameters
from opentelemetry.sdk.trace.export import BatchSpanProcessor
processor = BatchSpanProcessor(
OTLPSpanExporter(),
max_queue_size=2048, # Max spans in queue
max_export_batch_size=512, # Max spans per batch
schedule_delay_millis=5000, # Export every 5 seconds
export_timeout_millis=30000 # Timeout per export
)
3. Use SimpleSpanProcessor for low-throughput apps
from opentelemetry.sdk.trace.export import SimpleSpanProcessor
# Exports spans immediately — no buffering
processor = SimpleSpanProcessor(OTLPSpanExporter())
provider.add_span_processor(processor)
Warning: SimpleSpanProcessor exports synchronously, which adds latency to each span. Use only for development or very low throughput.
4. Handle graceful shutdown in web frameworks
from flask import Flask
from opentelemetry import trace
from opentelemetry.sdk.trace import TracerProvider
from opentelemetry.sdk.trace.export import BatchSpanProcessor
from opentelemetry.exporter.otlp.proto.grpc.trace_exporter import OTLPSpanExporter
app = Flask(__name__)
provider = TracerProvider()
processor = BatchSpanProcessor(OTLPSpanExporter())
provider.add_span_processor(processor)
trace.set_tracer_provider(provider)
@app.teardown_appcontext
def shutdown(exception=None):
processor.force_flush()
# Also register for process exit
import atexit
atexit.register(lambda: processor.shutdown())
5. Monitor processor metrics
from opentelemetry.sdk.trace.export import BatchSpanProcessor
processor = BatchSpanProcessor(OTLPSpanExporter())
# After test run
print(f"Queue size: {processor.queue.qsize()}")
print(f"Dropped spans: {processor.dropped_spans}")
processor.force_flush()
print("Flushed remaining spans")
Expected output:
Queue size: 0
Dropped spans: 0
Flushed remaining spans
Prevention Tips
- Register
processor.shutdown()withatexitfor graceful shutdown - Configure
schedule_delay_millisto match your latency requirements - Use
force_flush()before long idle periods - Monitor queue size to detect export bottlenecks
- Use
BatchSpanProcessorfor production,SimpleSpanProcessorfor development
Common Mistakes with batch processor
- Forgetting
deriving (Show, Eq)on custom data types needed for debugging - Placing the wildcard pattern first in case expressions, making all subsequent patterns unreachable
- Using
headandtailinstead of pattern matching, causing runtime errors on empty lists
These mistakes appear frequently in real-world OPENTELEMETRY code. DodaTech's contributors have identified these patterns through analysis of open-source projects and production systems.
Practice Exercise
Write a pure function that safely divides two integers using Maybe, then test it with edge cases like division by zero and negative numbers.
This exercise reinforces the concepts covered in this guide. Try implementing it before checking online solutions.
FAQ
Built by the developers of DodaTech
Doda Browser, DodaZIP & Durga Antivirus Pro