Webhook Flow — Complete Guide
In this tutorial, you will learn about Webhook Flow. We cover key concepts, practical examples, and best practices to help you master this topic.
Learn the complete webhook flow from event trigger to delivery: payload construction, HTTP POST transmission, response handling, retry logic, acknowledgment, and dead letter queues.
What You Learn
You will understand the end-to-end lifecycle of a webhook: how events trigger payloads, how payloads are delivered via HTTP, how consumers acknowledge receipt, how providers handle failures with retries, and what happens when all retry attempts are exhausted.
Why It Matters
Every webhook integration depends on this flow. Understanding it helps you debug delivery failures, design robust consumers, configure retry policies correctly, and build reliable event-driven systems.
Real-World Use
DodaTech's webhook dispatch system processes 500K Webhooks daily across 12000 registered endpoints. Understanding the flow from event creation to delivery acknowledgment is critical for maintaining 99.95% delivery success rate.
Flow Diagram
sequenceDiagram
participant Source as Event Source
participant Provider as Webhook Provider
participant Queue as Retry Queue
participant Consumer as Your Server
Source->>Provider: Event occurs
Provider->>Provider: Build payload
Provider->>Provider: Sign payload
Provider->>Consumer: POST /webhook (payload)
Consumer->>Consumer: Verify signature
Consumer->>Consumer: Process event
Consumer-->>Provider: 200 OK
Provider->>Provider: Mark delivered
The flow starts when an event occurs in the source system. The provider constructs the webhook payload, signs it, and sends it to your registered endpoint.
Step 1: Event Trigger
// Provider detects an event
class WebhookProvider {
constructor() {
this.subscribers = new Map(); // url -> { secret, events }
this.eventQueue = [];
}
// Called when an event occurs in the system
onEvent(eventType, eventData) {
console.log(`Event triggered: ${eventType}`);
// Find all subscribers for this event type
for (const [url, config] of this.subscribers) {
if (config.events.includes(eventType)) {
this.enqueueWebhook(url, config.secret, {
event: eventType,
data: eventData,
timestamp: new Date().toISOString(),
});
}
}
}
enqueueWebhook(url, secret, payload) {
this.eventQueue.push({ url, secret, payload });
this.processQueue();
}
}
Expected behavior: When an event occurs, the provider finds matching subscribers and queues webhooks for delivery. Events can be user registration, payment success, file upload, or any system action.
Step 2: Payload Construction
// Build the webhook payload
function buildWebhookPayload(eventType, eventData, secret) {
const payload = {
specversion: '1.0',
id: generateUniqueId(),
source: '/events/payments',
type: eventType,
datacontenttype: 'application/json',
time: new Date().toISOString(),
data: eventData,
};
// Add HMAC signature
const signature = signPayload(JSON.stringify(payload), secret);
payload.signature = signature;
return payload;
}
function generateUniqueId() {
return `wh_${Date.now()}_${Math.random().toString(36).slice(2, 10)}`;
}
function signPayload(payload, secret) {
const crypto = require('crypto');
return crypto
.createHmac('sha256', secret)
.update(payload)
.digest('hex');
}
Expected output: Webhook payload contains metadata (specversion, id, source, type, time) and the actual event data. The signature field allows consumers to verify authenticity.
Step 3: HTTP Delivery
// Provider sends the HTTP POST
async function deliverWebhook(url, payload, headers = {}) {
const startTime = Date.now();
try {
const response = await fetch(url, {
method: 'POST',
headers: {
'Content-Type': 'application/json',
'User-Agent': 'WebhookProvider/1.0',
'X-Webhook-ID': payload.id,
'X-Webhook-Signature': payload.signature,
'X-Webhook-Timestamp': payload.time,
...headers,
},
body: JSON.stringify(payload),
timeout: 10000, // 10 second timeout
});
const duration = Date.now() - startTime;
console.log(`Delivery to ${url}: ${response.status} in ${duration}ms`);
return {
success: response.status >= 200 && response.status < 300,
status: response.status,
duration,
};
} catch (err) {
const duration = Date.now() - startTime;
console.error(`Delivery failed to ${url}: ${err.message}`);
return {
success: false,
status: 0,
error: err.message,
duration,
};
}
}
Expected output: Provider sends HTTP POST to the consumer URL. Success is determined by a 2xx response within the timeout. Timeouts and network errors are treated as failures.
Step 4: Consumer Processing
// Consumer receives the webhook
const express = require('express');
const crypto = require('crypto');
const app = express();
app.use(express.json({
verify: (req, res, buf) => {
req.rawBody = buf.toString();
}
}));
// Store raw body for signature verification
app.post('/webhook', (req, res) => {
const {
'x-webhook-signature': signature,
'x-webhook-id': webhookId,
'x-webhook-timestamp': timestamp,
} = req.headers;
// Step 1: Verify signature
const expectedSig = crypto
.createHmac('sha256', process.env.WEBHOOK_SECRET)
.update(req.rawBody)
.digest('hex');
if (signature !== expectedSig) {
console.error(`Invalid signature for webhook ${webhookId}`);
return res.status(401).send('Invalid signature');
}
// Step 2: Check timestamp (prevent replay attacks)
const webhookTime = new Date(timestamp).getTime();
const now = Date.now();
if (now - webhookTime > 300000) { // 5 minutes tolerance
console.error(`Stale webhook ${webhookId}: ${timestamp}`);
return res.status(400).send('Stale webhook');
}
// Step 3: Check idempotency
if (processedIds.has(webhookId)) {
console.log(`Duplicate webhook ${webhookId}, already processed`);
return res.status(200).send('Already processed');
}
// Step 4: Acknowledge immediately
res.status(200).send('OK');
// Step 5: Process asynchronously
setImmediate(() => {
processWebhookAsync(req.body);
});
});
function processWebhookAsync(payload) {
processedIds.add(payload.id);
console.log(`Processing webhook ${payload.id}: ${payload.type}`);
// Actual business logic here
}
app.listen(3000);
Expected behavior: Consumer verifies signature, checks timestamp, idempotency, acknowledges immediately with 200, then processes the webhook asynchronously to avoid blocking the response.
Step 5: Retry Logic
// Provider retry with exponential backoff
class WebhookRetryManager {
constructor() {
this.retryDelays = [60, 300, 1800, 7200, 21600]; // 1min, 5min, 30min, 2hr, 6hr
this.maxRetries = this.retryDelays.length;
}
async deliverWithRetry(url, payload, attempt = 1) {
const result = await deliverWebhook(url, payload);
if (result.success) {
console.log(`Delivered on attempt ${attempt}`);
return { delivered: true, attempts: attempt };
}
if (attempt > this.maxRetries) {
console.error(`All ${this.maxRetries} attempts failed for ${url}`);
return { delivered: false, attempts: attempt };
}
const delayMs = this.retryDelays[attempt - 1] * 1000;
console.log(`Retry ${attempt}/${this.maxRetries} in ${delayMs/1000}s`);
await new Promise(r => setTimeout(r, delayMs));
return this.deliverWithRetry(url, payload, attempt + 1);
}
}
Expected output: First retry at 1 minute, then 5 minutes, 30 minutes, 2 hours, 6 hours. After 5 retries over ~9 hours, the webhook moves to dead letter queue.
Step 6: Dead Letter Queue
// Dead letter queue for permanently failed webhooks
class DeadLetterQueue {
constructor() {
this.queue = [];
this.maxEntries = 1000;
}
add(webhook, error) {
const entry = {
webhook,
error,
failedAt: new Date().toISOString(),
attempts: webhook.attempts,
};
this.queue.push(entry);
if (this.queue.length > this.maxEntries) {
this.queue.shift();
}
// Alert operations team
console.error(`Webhook dead lettered: ${webhook.id}`);
this.sendAlert(entry);
}
sendAlert(entry) {
// Send to Slack, email, or PagerDuty
console.log('Alert sent for dead lettered webhook');
}
retryDeadLetter(webhookId) {
const index = this.queue.findIndex(
e => e.webhook.id === webhookId
);
if (index === -1) return false;
const entry = this.queue[index];
this.queue.splice(index, 1);
// Re-deliver with fresh retry count
deliverWebhook(entry.webhook.url, entry.webhook.payload);
return true;
}
getStats() {
return {
total: this.queue.length,
oldest: this.queue[0]?.failedAt,
newest: this.queue[this.queue.length - 1]?.failedAt,
};
}
}
Expected output: Failed webhooks are stored in the dead letter queue with error details and timestamps. Operations can retry them manually. Alerts notify the team of persistent failures.
Common Mistakes
1. Processing Before Acknowledging
Doing database writes or API calls before returning 200 causes timeouts and retries. Acknowledge receipt within seconds. Process asynchronously using a task queue.
2. Invalid Signature Comparison
Comparing HMAC hex digests requires case-insensitive comparison or consistent case. Some providers use uppercase, others lowercase. Normalize both sides.
3. No Idempotency Check
Without idempotency, retries cause duplicate processing: charging a customer twice, creating duplicate orders, sending duplicate emails. Use webhook ID to detect and skip duplicates.
4. Blocking Event Loop (Node.js)
CPU-heavy webhook processing blocks other requests. Use worker threads or queue the work. Return 200 immediately, process in the background.
5. Ignoring Dead Letter Queue
Failed webhooks accumulate silently. Monitor dead letter queue size. Alert when it grows beyond threshold. Investigate and retry failed webhooks daily.
Practice Questions
1. What happens if the consumer returns a 5xx response?
The provider treats it as a delivery failure and retries with exponential backoff. The webhook is retried up to the maximum retry count before dead lettering.
2. Why is idempotency important in webhook flow?
Without idempotency, retried webhooks cause duplicate processing. Idempotency keys allow the consumer to detect and skip already-processed webhooks.
3. What is the purpose of the timestamp in webhook headers?
Timestamps prevent replay attacks. The consumer checks that the webhook is recent (within 5 minutes). Old webhooks are rejected even if the signature is valid.
4. How do dead letter queues help operations?
They store permanently failed webhooks for manual inspection and retry. Operations can debug the failure, fix the issue, and replay the webhook without losing data.
Challenge
Build a complete webhook flow simulation: provider generates events for 100 random subscribers, delivers with retry logic, tracks delivery success rate, logs all attempts, and reports dead letter queue statistics after 10 minutes of simulation.
FAQ
Mini Project: Webhook Flow Dashboard
Build a dashboard showing real-time webhook flow: events being triggered, payloads being delivered, response status codes, retry attempts, and dead letter queue status. Use Server-Sent Events to push updates to the dashboard in real time.
What's Next
Now that you understand the complete webhook flow, learn about Payload Format standards and how to structure webhook data effectively.
Built by the developers of DodaTech
Doda Browser, DodaZIP & Durga Antivirus Pro