Webhooks Complete Guide: Event-Driven Server-to-Server Communication

In this tutorial, you'll learn about webhooks: HTTP callbacks that enable real-time event-driven communication between web applications by sending automated notifications when specific events occur.

Webhooks are user-defined HTTP callbacks that enable real-time push-based communication between web applications, triggered automatically when specific events occur on the provider side.

What You'll Learn

Webhook architecture and the push vs polling tradeoff
Building webhook providers and consumers
Payload signing and signature verification (HMAC-SHA256)
Retry policies, idempotency, and dead-letter queues
Rate Limiting, monitoring, and security best practices

Why Webhooks Matter

Polling wastes bandwidth and server resources — clients repeatedly check for updates even when nothing changed. Webhooks flip this: the server pushes data the instant an event happens. DodaTech's Durga Antivirus Pro uses webhooks to notify partner SIEM systems about threat detections in real time, eliminating the need for partners to poll a status endpoint while ensuring no threat event is missed.

flowchart LR
    subgraph "Webhook Flow"
        A["Event Occurs\n(threat detected)"] --> B["Webhook Provider\n(Durga Antivirus)"]
        B --> C["Sign Payload\n(HMAC-SHA256)"]
        C --> D["POST to Consumer URL"]
        D --> E["Consumer Endpoint\n(POST /webhook)"]
        E -->|"200 OK"| F["Delivery Success"]
        E -->|"4xx/5xx"| G["Retry Queue\n(exp backoff)"]
        G --> D
        G -->|"Max retries"| H["Dead Letter Queue"]
    end
    style B fill:#dbeafe,stroke:#2563eb
    style E fill:#fef3c7,stroke:#d97706
    style G fill:#fca5a5,stroke:#dc2626

ℹ️ Info

Prerequisites: HTTP knowledge, JSON familiarity, and basic programming skills in at least one language (JavaScript, Python, or Java).

Webhooks vs Polling

Aspect	Webhooks	Polling
Direction	Server pushes	Client pulls
Latency	Real-time (ms)	Depends on interval (seconds-minutes)
Server load	Event-triggered	Continuous (even when nothing changes)
Bandwidth	Only when events occur	Every poll interval
Complexity	Higher (retries, signing, scaling)	Lower
Best for	Time-sensitive notifications	Non-critical status checks

Common Mistakes

1. Not Verifying Signatures

Without HMAC signature verification, anyone can send fake webhooks to your endpoint. Always verify signatures using a shared secret.

2. Blocking on Webhook Processing

Webhook senders expect fast responses (2-5 seconds). Queue heavy processing to a background task and return 200 immediately to avoid timeouts.

3. Ignoring Idempotency

Network failures cause duplicate webhook deliveries. Use idempotency keys (event IDs) to detect and safely skip duplicate events.

4. No Retry Logic

Webhooks fail — networks drop, servers restart. Implement exponential backoff with 3-5 retry attempts before moving to a dead-letter queue.

5. Not Returning Proper HTTP Status Codes

Return 200 on success. Return 4xx for bad payloads (no retry) or 5xx for temporary issues (triggers retry). Proper codes prevent unnecessary retries or missed deliveries.

Practice Questions

What is the main advantage of webhooks over polling?
How do you verify a webhook payload is authentic?
What HTTP status code should a webhook consumer return on success?
What is the purpose of a dead-letter queue in webhook delivery?
How do idempotency keys prevent duplicate processing?

Answers:

Webhooks push data in real time when events occur, eliminating the need for clients to poll. This reduces bandwidth, latency, and server load.
The sender signs the payload with HMAC-SHA256 using a shared secret. The receiver computes the expected signature and compares using hmac.compare_digest to prevent timing attacks.
200 OK. Any 4xx (client error) or 5xx (server error) signals failure and may trigger retries according to the provider's retry policy.
A dead-letter queue stores events that failed after exhausting all retry attempts. It enables manual inspection, replay, and debugging of undeliverable webhooks.
Each event includes a unique idempotency key. The consumer checks if a key was already processed; if so, it skips processing and returns the previous result. This prevents double charges or duplicate actions.

Challenge: Design a webhook system for DodaTech's file scanning service. When a scan completes, notify the user's dashboard (via Websocket) and an external SIEM system (via webhook). Include HMAC signing, retries with exponential backoff, idempotency using scan IDs, and a dead-letter queue for failed deliveries.

FAQ

What is the difference between webhooks and APIs?

APIs follow a request-response pattern — you ask, you receive. Webhooks follow an event-callback pattern — when something happens, the server notifies you. APIs are pull-based; webhooks are push-based.

Can webhooks send binary data?

Yes, webhooks can send any content type, but JSON is most common. For binary data, use base64 encoding within JSON or multipart/form-data.

What happens if my webhook endpoint is down?

The sender retries with exponential backoff (typically 3-5 attempts over 24-72 hours). After exhausting retries, the event is logged as failed or moved to a dead-letter queue.

How do webhooks scale?

Webhooks scale horizontally behind a load balancer. The sender distributes webhooks across registered URLs. Use a message queue (RabbitMQ, Redis) to buffer deliveries during traffic spikes.

Should webhooks be synchronous or asynchronous?

Always process webhooks asynchronously. Return 200 immediately, then process the event in a background queue. Synchronous processing blocks the sender and increases timeout failures.

Try It Yourself

# Test receiving webhooks locally with ngrok
ngrok http 3000

# Forward webhook payloads to your local server
# ngrok gives you a public URL like https://abc123.ngrok.io
# Use that URL in your webhook provider configuration

What's Next

Topic	Description
Introduction to Webhooks	First steps with webhooks
AsyncAPI Specification	Documenting event-driven APIs
WebSocket Guide	Bidirectional real-time communication
Server-Sent Events	One-way server push

➡ Introduction to Webhooks

➡ AsyncAPI Specification

Published Topics

Monitoring Delivery