Publish-Subscribe Pattern — Complete Guide

In this tutorial, you will learn about Publish. We cover key concepts, practical examples, and best practices to help you master this topic.

Publish-subscribe delivers each message to all subscribed consumers simultaneously, enabling event broadcasting for real-time notifications and fan-out workflows across multiple services.

What You'll Learn

By the end of this lesson, you will understand the publish-subscribe pattern, when to use it over point-to-point, how to implement it with RabbitMQ fanout exchanges, and how to manage subscriptions.

Why It Matters

Modern applications rarely consist of a single service. When a user registers, multiple services need to react: send a welcome email, create a profile, initialize analytics tracking, and notify the CRM. Pub-sub broadcasts one event to all interested services without the producer knowing about any of them.

Real-World Use

A streaming platform publishes a "new video uploaded" event. Three services subscribe: the transcoding service starts encoding, the notification service sends alerts to subscribers, and the recommendation service updates its model. Adding a fourth subscriber requires zero changes to the producer.

How Pub-Sub Works

flowchart LR
    P[Publisher] --> EX[Exchange / Topic]
    EX --> Q1[Queue: Email Service]
    EX --> Q2[Queue: Analytics]
    EX --> Q3[Queue: Notification]
    Q1 --> C1[Email Consumer]
    Q2 --> C2[Analytics Consumer]
    Q3 --> C3[Notification Consumer]
    style EX fill:#f90,color:#fff

A publisher sends a message to an exchange. The exchange fans out copies to every bound queue. Each consumer receives its own copy of the message.

Implementing with RabbitMQ Fanout Exchange

import pika
import json

connection = pika.BlockingConnection(pika.ConnectionParameters('localhost'))
channel = connection.channel()

channel.exchange_declare(exchange='events', exchange_type='fanout')

message = json.dumps({'event': 'user.registered', 'email': 'alice@example.com'})
channel.basic_publish(exchange='events', routing_key='', body=message)
print("Published event to fanout exchange 'events'")
connection.close()

Expected output:

Published event to fanout exchange 'events'

import pika

connection = pika.BlockingConnection(pika.ConnectionParameters('localhost'))
channel = connection.channel()

channel.exchange_declare(exchange='events', exchange_type='fanout')

result = channel.queue_declare(queue='', exclusive=True)
queue_name = result.method.queue
print(f"Created exclusive queue: {queue_name}")

channel.queue_bind(exchange='events', queue=queue_name)

def callback(ch, method, properties, body):
    print(f"Received: {body.decode()}")

channel.basic_consume(queue=queue_name, on_message_callback=callback, auto_ack=True)
print("Waiting for events. Press Ctrl+C to exit.")
channel.start_consuming()

Expected output:

Created exclusive queue: amq.gen-abc123
Waiting for events. Press Ctrl+C to exit.
Received: {"event": "user.registered", "email": "alice@example.com"}

Each consumer creates an exclusive queue and binds it to the fanout exchange. The exchange copies every message to all bound queues.

The Analogy of Radio Broadcasting

Think of pub-sub like a radio station. The station broadcasts music on a specific frequency. Anyone with a radio tuned to that frequency receives the broadcast. The station does not know how many radios are listening, and it does not care. Listeners can tune in or out at any time.

In messaging terms, the exchange is the radio tower, queues are the radios, and consumers are the listeners.

Point-to-Point vs Pub-Sub at a Glance

Common Mistakes

1. Assuming Delivery Guarantees

If a consumer is offline when a message is published, it misses the message unless it has a durable queue with bindings. Exclusive queues are deleted when the consumer disconnects, so messages sent during downtime are lost.

2. Using Pub-Sub for Task Distribution

Pub-sub sends every message to every subscriber. If you want one worker to Process each task, use point-to-point instead. Pub-sub duplicates work unnecessarily.

3. Binding the Same Queue Multiple Times

Binding the same queue to the same exchange multiple times has no effect in RabbitMQ. Messages are not duplicated. Only unique queue-exchange bindings create separate message copies.

4. Not Considering Message Volume

Every subscriber receives every message. With 10 subscribers, each message creates 10 copies. For high-throughput systems, this multiplies network and storage requirements.

5. Mixing Durable and Exclusive Queues

Durable queues survive broker restarts. Exclusive queues are deleted when the consumer disconnects. Decide carefully which type each subscriber needs based on whether missed messages during downtime are acceptable.

Practice Questions

1. How does pub-sub differ from point-to-point?

Pub-sub delivers each message to all subscribers. Point-to-point delivers each message to exactly one consumer. Pub-sub is for event broadcasting; point-to-point is for task distribution.

2. What exchange type does RabbitMQ use for pub-sub?

Fanout exchange. It ignores routing keys and broadcasts every message to all bound queues.

3. Do subscribers receive messages published before they subscribed?

With exclusive queues, no. With durable queues that existed before the subscriber connected, messages accumulate while the subscriber is offline and are delivered on reconnection.

4. How do you add a new subscriber?

Create a new queue, bind it to the exchange, and start a consumer. No changes to the publisher or other subscribers are needed. This is the key benefit of pub-sub.

Challenge

Design a pub-sub system for an e-commerce platform: "order.placed" event triggers inventory deduction, payment processing, email confirmation, and analytics tracking. Four separate services subscribe. Implement graceful handling for when a subscriber is temporarily down.

FAQ

Mini Project: Event Notification System

import pika
import json
import threading
import time

def start_subscriber(name, exchange):
    connection = pika.BlockingConnection(pika.ConnectionParameters('localhost'))
    channel = connection.channel()
    channel.exchange_declare(exchange=exchange, exchange_type='fanout')
    result = channel.queue_declare(queue='', exclusive=True)
    queue_name = result.method.queue
    channel.queue_bind(exchange=exchange, queue=queue_name)

    def callback(ch, method, properties, body):
        event = json.loads(body)
        print(f"[{name}] Received: {event['type']} — {event['data']}")

    channel.basic_consume(queue=queue_name, on_message_callback=callback, auto_ack=True)
    channel.start_consuming()

services = ['Email', 'Analytics', 'CRM', 'Search']
threads = []
for svc in services:
    t = threading.Thread(target=start_subscriber, args=(svc, 'app_events'), daemon=True)
    t.start()
    threads.append(t)

time.sleep(1)

connection = pika.BlockingConnection(pika.ConnectionParameters('localhost'))
channel = connection.channel()
channel.exchange_declare(exchange='app_events', exchange_type='fanout')

events = [
    {'type': 'user.registered', 'data': 'alice@example.com'},
    {'type': 'order.placed', 'data': 'ORD-1234'},
    {'type': 'payment.received', 'data': '$49.99'},
]

for event in events:
    channel.basic_publish(exchange='app_events', routing_key='', body=json.dumps(event))
    print(f"[Publisher] Sent: {event['type']}")

connection.close()
time.sleep(1)

Expected output:

[Publisher] Sent: user.registered
[Email] Received: user.registered — alice@example.com
[Analytics] Received: user.registered — alice@example.com
[CRM] Received: user.registered — alice@example.com
[Search] Received: user.registered — alice@example.com
[Publisher] Sent: order.placed
[Email] Received: order.placed — ORD-1234
...

What's Next

Now that you understand pub-sub, explore message broker concepts for a deeper look at how brokers like RabbitMQ and Kafka handle routing, then dive into producer-consumer patterns for advanced workflow design.