Message TTL — Complete Guide

In this tutorial, you will learn about Message TTL. We cover key concepts, practical examples, and best practices to help you master this topic.

Message TTL (time-to-live) sets an expiration time for messages. Expired messages are discarded or routed to a dead letter exchange for inspection.

Per-Queue vs Per-Message TTL

Two ways to set TTL:

1. Per-queue: All messages in the queue expire after a fixed time
2. Per-message: Each message has its own expiration time

Queue-Level TTL

import pika
import time

connection = pika.BlockingConnection(pika.ConnectionParameters('localhost'))
channel = connection.channel()

args = {
    'x-message-ttl': 5000,
    'x-dead-letter-exchange': 'dlx',
}
channel.queue_declare(queue='ttl_queue', durable=True, arguments=args)

channel.basic_publish(
    exchange='', routing_key='ttl_queue',
    body='This message expires in 5 seconds',
    properties=pika.BasicProperties(delivery_mode=2)
)
print("Message published with 5s TTL")
print(f"Current time: {time.strftime('%H:%M:%S')}")

time.sleep(7)

method_frame, _, body = channel.basic_get(queue='ttl_queue', auto_ack=True)
if method_frame:
    print(f"Message still alive: {body.decode() if body else 'None'}")
else:
    print("Message expired (no message in queue)")

connection.close()

Expected output:

Message published with 5s TTL
Current time: 10:00:00
Message expired (no message in queue)

After 5 seconds, the message expires. If a DLX is configured, it is routed there. Otherwise, it is discarded.

Per-Message TTL

import pika
import time

connection = pika.BlockingConnection(pika.ConnectionParameters('localhost'))
channel = connection.channel()
channel.queue_declare(queue='mixed_ttl', durable=True)

messages = [
    ('Expires in 2s', 2000),
    ('Expires in 10s', 10000),
    ('No expiration', None),
]

for body, ttl in messages:
    props = pika.BasicProperties(
        delivery_mode=2,
        expiration=str(ttl) if ttl else None
    )
    channel.basic_publish(exchange='', routing_key='mixed_ttl', body=body, properties=props)
    ttl_str = f'{ttl}ms' if ttl else 'none'
    print(f"Published: '{body}' (TTL: {ttl_str})")

connection.close()

Expected output:

Published: 'Expires in 2s' (TTL: 2000ms)
Published: 'Expires in 10s' (TTL: 10000ms)
Published: 'No expiration' (TTL: none)

When Does TTL Start?

TTL starts when the message reaches the queue, not when it is published. If a message sits in a producer buffer or a routing delay, the TTL clock has not started yet.

TTL and Dead Lettering

When a message expires, RabbitMQ checks if the queue has a DLX configured. If yes, the expired message is routed to the DLX. If no DLX, the message is discarded.

args = {
    'x-message-ttl': 30000,
    'x-dead-letter-exchange': 'dlx',
}

This configuration expires messages after 30 seconds and moves them to the DLX for inspection.

Common Mistakes

1. Setting TTL Too Short

If TTL is shorter than the expected processing time, messages expire in the queue before consumers can Process them. Measure processing time and set TTL to at least 2x the p99 processing time.

2. Confusing Milliseconds with Seconds

TTL values are in milliseconds. x-message-ttl: 5000 = 5 seconds, not 5 milliseconds. A common source of prematurely expired messages.

3. Not Setting DLX with TTL

Expired messages are silently discarded without a DLX. Always configure DLX if you need to inspect or replay expired messages.

4. Per-Message TTL Exceeding Queue TTL

If both per-message and per-queue TTL are set, the shorter TTL wins. Per-queue TTL overrides slower per-message TTL.

5. Using TTL for Exactly-Once Deadlines

TTL in RabbitMQ is approximate, not precise. Messages may expire a few seconds late under load. Do not rely on TTL for strict deadlines.

Practice Questions

1. What is the difference between per-queue and per-message TTL?

Per-queue TTL applies to all messages in the queue. Per-message TTL is set individually on each message. Per-queue TTL is enforced even if per-message TTL is longer.

2. When does TTL start counting?

When the message reaches the queue (enqueued), not when it is published by the producer. Time spent in producer buffers or network transit does not count.

3. What happens to expired messages?

They are discarded unless a dead letter exchange is configured. With a DLX, expired messages are routed there.

4. What unit is TTL specified in?

Milliseconds. 5000 = 5 seconds. 60000 = 1 minute. 3600000 = 1 hour.

Challenge

Design a TTL Strategy for a password reset system. Reset tokens expire after 15 minutes. Messages older than 15 minutes should be routed to a DLX for audit logging. Handle the case where a message reaches the queue just before the 15-minute deadline.

FAQ

Mini Project: TTL-Aware Consumer

import pika, json, time

def setup_ttl_queues():
    conn = pika.BlockingConnection(pika.ConnectionParameters('localhost'))
    ch = conn.channel()

    ch.exchange_declare(exchange='orders_dlx', exchange_type='fanout', durable=True)
    ch.queue_declare(queue='expired_orders', durable=True)
    ch.queue_bind(exchange='orders_dlx', queue='expired_orders')

    args = {
        'x-message-ttl': 10000,
        'x-dead-letter-exchange': 'orders_dlx',
    }
    ch.queue_declare(queue='orders', durable=True, arguments=args)

    ch.basic_publish(exchange='', routing_key='orders', body='Order #123',
                     properties=pika.BasicProperties(delivery_mode=2))
    print("Order published with 10s TTL")
    conn.close()

def check_dlq():
    time.sleep(12)
    conn = pika.BlockingConnection(pika.ConnectionParameters('localhost'))
    ch = conn.channel()

    method, _, body = ch.basic_get(queue='expired_orders', auto_ack=True)
    if method:
        print(f"Expired order found in DLQ: {body.decode()}")
    else:
        print("No expired orders")

    conn.close()

setup_ttl_queues()
check_dlq()

Expected output:

Order published with 10s TTL
Expired order found in DLQ: Order #123

What's Next

Now that you understand message TTL, explore queue TTL for auto-deleting idle queues, then learn about publisher confirms for reliable message publishing.