Queue Mirroring in RabbitMQ — Complete Guide

In this tutorial, you will learn about Queue Mirroring in RabbitMQ. We cover key concepts, practical examples, and best practices to help you master this topic.

Queue mirroring replicates queue contents across cluster nodes, ensuring high availability and preventing message loss when a node fails.

What You Learn

You will learn how classic queue mirroring works, how to configure mirrored queues, how quorum queues differ from mirrored queues, and best practices for high availability.

Why It Matters

In a cluster without mirroring, each queue lives on one node. If that node fails, the queue and its messages are lost. Mirroring ensures a copy exists on another node, providing automatic failover.

Real-World Use

Durga Antivirus Pro uses quorum queues for critical signature update queues. If a node fails during an update broadcast, another node takes over without losing a single signature message.

How Queue Mirroring Works

flowchart TB
    subgraph "Cluster"
        N1[Node 1
Queue Master]
        N2[Node 2
Mirror 1]
        N3[Node 3
Mirror 2]
        N1 -->|replicate| N2
        N1 -->|replicate| N3
    end
    P[Producer] -->|publish| N1
    C[Consumer] -->|consume| N1
    style N1 fill:#f90,color:#fff

One node is the master. Other nodes are mirrors. All publishing and consuming goes through the master, which replicates to all mirrors synchronously.

Configuring Mirrored Queues via Policy

Set a policy to mirror queues matching a pattern:

# Mirror all queues with "ha-" prefix across all nodes
sudo rabbitmqctl set_policy ha-all "^ha-" '{"ha-mode":"all","ha-sync-mode":"automatic"}'

# Mirror important queues across 2 nodes
sudo rabbitmqctl set_policy ha-two "^critical-" '{"ha-mode":"exactly","ha-params":2,"ha-sync-mode":"automatic"}'

import pika

conn = pika.BlockingConnection(pika.ConnectionParameters('localhost'))
ch = conn.channel()

# Queue arguments for mirroring
args = {
    'x-ha-policy': 'all'
}

ch.queue_declare(
    queue='mirrored_queue',
    durable=True,
    arguments=args
)

ch.basic_publish(exchange='', routing_key='mirrored_queue', body='mirror me')
print("Message published to mirrored queue")

# Check which node is master
queue_info = ch.queue_declare(queue='mirrored_queue', passive=True)
print(f"Queue: {queue_info.method.queue}")
print(f"Messages: {queue_info.method.message_count}")

conn.close()

Quorum Queues (Recommended)

Quorum queues are the modern alternative to classic mirrored queues. They use Raft consensus for Replication:

import pika

conn = pika.BlockingConnection(pika.ConnectionParameters('localhost'))
ch = conn.channel()

args = {
    'x-queue-type': 'quorum',
    'x-quorum-initial-group-size': 3
}

ch.queue_declare(
    queue='quorum_queue',
    durable=True,
    arguments=args
)

ch.basic_publish(exchange='', routing_key='quorum_queue', body='quorum test')
print("Published to quorum queue")

ch.queue_delete(queue='quorum_queue')
conn.close()

Classic Mirrored vs Quorum Queues

Synchronization Modes

# Automatic sync (recommended)
sudo rabbitmqctl set_policy ha-sync "^sync-" '{"ha-mode":"all","ha-sync-mode":"automatic"}'

# Manual sync
sudo rabbitmqctl set_policy ha-sync-manual "^manual-" '{"ha-mode":"all","ha-sync-mode":"manual"}'

With manual sync, mirrors sync only when you explicitly trigger:

sudo rabbitmqctl sync_queue queue_name

Monitoring Mirrored Queues

# Check which queues are mirrored
sudo rabbitmqctl list_queues name slave_nodes synchronised

# Check queue status
sudo rabbitmqctl list_queues name type master_pid slave_pids

Expected output:

name	slave_nodes	synchronised
mirrored_queue	[rabbit@node2,rabbit@node3]	true

Common Mistakes

1. Mirroring All Queues Unnecessarily

Mirroring adds write overhead. Only mirror critical queues. Use policy patterns to selectively mirror.

2. Not Setting ha-sync-mode

Without automatic sync, new mirrors may not have all messages. Manual sync is error-prone in production.

3. Using Classic Mirrored Queues for New Deployments

Quorum queues are more reliable and have better data safety guarantees. Use quorum queues for all new critical workloads.

4. Ignoring the Performance Impact of Many Mirrors

Each mirror adds write latency because the master waits for all mirrors to acknowledge. Use ha-mode: exactly with 2-3 mirrors instead of all.

5. Not Testing Failover

Mirroring only helps if failover works. Test by killing the master node and verifying consumers reconnect and continue processing.

Practice Questions

1. What is the purpose of queue mirroring?

To replicate queue contents across multiple cluster nodes so that messages survive individual node failures.

2. How does a quorum queue differ from a classic mirrored queue?

Quorum queues use Raft consensus for strong consistency and automatic leader election. Classic mirrored queues use master-slave replication with manual failover.

3. What is ha-mode exactly used for?

It limits the number of mirrors to a specific count. For example, ha-mode=exactly with ha-params=2 means exactly 2 nodes have copies.

4. When should you use automatic vs manual sync?

Use automatic sync in production. Manual sync is useful when adding a new mirror to avoid network overhead during sync.

Challenge

Design a RabbitMQ HA Strategy for a payment processing system. The system requires no message loss, low latency, and must survive two simultaneous node failures. Choose between mirrored queues, quorum queues, and streaming queues. Justify your choice.

FAQ

Mini Project: HA Queue Setup

#!/bin/bash
# Set up high-availability queues on an existing RabbitMQ cluster

# 1. Enable quorum queues (built-in, no plugin needed)

# 2. Create policy for critical queues
sudo rabbitmqctl set_policy critical "^critical-" '{
    "queue-type": "quorum",
    "ha-mode": "exactly",
    "ha-params": 3,
    "delivery-limit": 10
}' --priority 10 --apply-to queues

# 3. Create policy for mirrored regular queues
sudo rabbitmqctl set_policy important "^important-" '{
    "ha-mode": "exactly",
    "ha-params": 2,
    "ha-sync-mode": "automatic"
}' --priority 5 --apply-to queues

# 4. Verify policies
sudo rabbitmqctl list_policies

# 5. Test by declaring a critical queue
python3 -c "
import pika
conn = pika.BlockingConnection(pika.ConnectionParameters('localhost'))
ch = conn.channel()
args = {'x-queue-type': 'quorum'}
ch.queue_declare(queue='critical-payments', durable=True, arguments=args)
print('Quorum queue created')
props = ch.queue_declare(queue='critical-payments', passive=True)
print(f'Queue type: {props.arguments.get(\"x-queue-type\")}')
conn.close()
"

Expected output:

Quorum queue created
Queue type: quorum

What's Next

Now that you understand queue mirroring, explore the RabbitMQ management UI for monitoring queues and cluster health, then learn about RabbitMQ with Python for application integration.