Mini Project: Webhook Relay Service
In this tutorial, you will learn about Mini Project: Webhook Relay Service. We cover key concepts, practical examples, and best practices to help you master this topic.
Build a webhook relay service that receives webhooks from multiple providers, transforms them to a standard format, routes to multiple consumers, and provides delivery monitoring and retry capabilities.
What You Learn
You will build a complete webhook relay service that acts as an intermediary between webhook providers and your internal services. The relay handles provider-specific verification, payload transformation, consumer routing, delivery with retries, and monitoring.
Why It Matters
Direct webhook integration with every provider creates coupling. Each provider has different payload formats, verification methods, and delivery expectations. A relay service standardizes webhooks internally, decouples providers from consumers, and provides a single point for monitoring and retry.
Real-World Use
DodaTech's webhook relay service handles webhooks from Stripe, GitHub, SendGrid, and 12 other providers. The relay normalizes all webhooks to Standard Webhooks format, routes to 8 internal services, and provides unified monitoring. Adding a new provider takes hours instead of days.
System Architecture
graph TD
Stripe[Stripe] --> Relay[Webhook Relay]
GitHub[GitHub] --> Relay
SendGrid[SendGrid] --> Relay
Custom[Custom Provider] --> Relay
Relay --> Verify[Verification Layer]
Verify --> Transform[Transformation Layer]
Transform --> Router[Routing Layer]
Router --> Consumer1[Order Service]
Router --> Consumer2[Notification Service]
Router --> Consumer3[Analytics Service]
Router --> DLQ[Dead Letter Queue]
Router --> Monitor[Monitoring]
The relay receives webhooks from external providers, verifies provider-specific signatures, transforms payloads to a standard internal format, routes to the appropriate internal service, and monitors delivery.
Provider Handlers
// Provider handler registry
class ProviderHandlerRegistry {
constructor() {
this.handlers = new Map();
}
register(providerName, handler) {
this.handlers.set(providerName, handler);
console.log(`Registered provider handler: ${providerName}`);
}
getHandler(providerName) {
const handler = this.handlers.get(providerName);
if (!handler) throw new Error(`Unknown provider: ${providerName}`);
return handler;
}
getAllProviders() {
return Array.from(this.handlers.keys());
}
}
// Provider handler interface
class ProviderHandler {
constructor(name) {
this.name = name;
}
async verify(req) {
throw new Error('verify() must be implemented');
}
async transform(payload) {
throw new Error('transform() must be implemented');
}
getEventType(payload) {
throw new Error('getEventType() must be implemented');
}
}
// Stripe handler
class StripeHandler extends ProviderHandler {
constructor() {
super('stripe');
this.secret = process.env.STRIPE_WEBHOOK_SECRET;
}
async verify(req) {
const stripe = require('stripe')();
const sig = req.headers['stripe-signature'];
return stripe.webhooks.constructEvent(req.rawBody, sig, this.secret);
}
async transform(event) {
return {
id: event.id,
type: event.type,
timestamp: new Date(event.created * 1000).toISOString(),
data: event.data.object,
provider: 'stripe',
};
}
getEventType(payload) {
return payload.type;
}
}
// GitHub handler
class GitHubHandler extends ProviderHandler {
constructor() {
super('github');
this.secret = process.env.GITHUB_WEBHOOK_SECRET;
}
async verify(req) {
const signature = req.headers['x-hub-signature-256'];
const rawBody = req.rawBody;
const expectedSig = signature.replace('sha256=', '');
const computedSig = crypto
.createHmac('sha256', this.secret)
.update(rawBody)
.digest('hex');
if (!crypto.timingSafeEqual(
Buffer.from(computedSig),
Buffer.from(expectedSig)
)) {
throw new Error('Invalid GitHub signature');
}
return JSON.parse(rawBody);
}
async transform(payload) {
return {
id: payload.after || `gh_${Date.now()}`,
type: `github.${payload.action || 'push'}`,
timestamp: new Date().toISOString(),
data: payload,
provider: 'github',
};
}
getEventType(payload) {
return payload.action || 'push';
}
}
const registry = new ProviderHandlerRegistry();
registry.register('stripe', new StripeHandler());
registry.register('github', new GitHubHandler());
Expected output: Provider handlers encapsulate provider-specific logic. Each handler implements verify(), transform(), and getEventType(). Adding a new provider means creating a new handler class.
Relay Server
const express = require('express');
const crypto = require('crypto');
class WebhookRelay {
constructor(options = {}) {
this.port = options.port || 3000;
this.registry = options.registry;
this.router = options.router;
this.monitor = options.monitor;
this.app = express();
this.setupMiddleware();
this.setupRoutes();
}
setupMiddleware() {
// Capture raw body
this.app.use('/relay/:provider', (req, res, next) => {
const chunks = [];
req.on('data', chunk => chunks.push(chunk));
req.on('end', () => {
req.rawBody = Buffer.concat(chunks).toString();
try {
req.body = JSON.parse(req.rawBody);
} catch (e) {
req.body = {};
}
next();
});
});
}
setupRoutes() {
this.app.post('/relay/:provider', async (req, res) => {
const startTime = Date.now();
const { provider } = req.params;
try {
const handler = this.registry.getHandler(provider);
// 1. Verify provider-specific signature
const verifiedPayload = await handler.verify(req);
this.monitor.recordVerification(provider, true);
// 2. Transform to standard format
const standardPayload = await handler.transform(verifiedPayload);
this.monitor.recordTransformation(provider, standardPayload.type);
// 3. Route to consumers
const routingResults = await this.router.route(standardPayload);
// 4. Record delivery
this.monitor.recordDelivery(provider, standardPayload, routingResults);
// 5. Acknowledge
const duration = Date.now() - startTime;
res.status(200).json({
status: 'accepted',
relayId: standardPayload.id,
durationMs: duration,
consumers: routingResults.length,
});
} catch (err) {
this.monitor.recordError(provider, err);
console.error(`Relay error [${provider}]:`, err.message);
const statusCode = err.message.includes('signature') ? 401 : 500;
res.status(statusCode).json({
error: 'Relay processing failed',
message: err.message,
});
}
});
}
start() {
this.app.listen(this.port, () => {
console.log(`Webhook relay on port ${this.port}`);
console.log(`Providers: ${this.registry.getAllProviders().join(', ')}`);
});
}
}
Expected output: Relay server accepts webhooks at /relay/:provider, verifies provider-specific signatures, transforms payloads, routes to consumers, and returns delivery status.
Router and Consumers
// Route webhooks to internal consumers
class WebhookRouter {
constructor() {
this.routes = new Map(); // eventType prefix -> consumer URLs
this.deliveryService = new DeliveryService();
}
addRoute(eventTypePrefix, consumerUrl, options = {}) {
if (!this.routes.has(eventTypePrefix)) {
this.routes.set(eventTypePrefix, []);
}
this.routes.get(eventTypePrefix).push({
url: consumerUrl,
options,
});
console.log(`Route: ${eventTypePrefix} -> ${consumerUrl}`);
}
async route(standardPayload) {
const results = [];
const eventType = standardPayload.type;
for (const [prefix, consumers] of this.routes) {
if (eventType.startsWith(prefix)) {
for (const consumer of consumers) {
const result = await this.deliveryService.deliver(
consumer.url,
standardPayload,
consumer.options
);
results.push({ url: consumer.url, ...result });
}
}
}
return results;
}
}
// Delivery service with retry
class DeliveryService {
async deliver(url, payload, options = {}) {
const maxRetries = options.maxRetries || 3;
for (let attempt = 1; attempt <= maxRetries; attempt++) {
try {
const response = await fetch(url, {
method: 'POST',
headers: {
'Content-Type': 'application/json',
'X-Relay-ID': payload.id,
'X-Relay-Provider': payload.provider,
},
body: JSON.stringify(payload),
signal: AbortSignal.timeout(10000),
});
if (response.ok) {
return { success: true, attempt };
}
// Don't retry client errors
if (response.status >= 400 && response.status < 500 &&
response.status !== 429) {
return { success: false, status: response.status, attempt };
}
} catch (err) {
if (attempt === maxRetries) {
return { success: false, error: err.message, attempt };
}
}
await new Promise(r => setTimeout(r, 1000 * Math.pow(2, attempt)));
}
return { success: false, error: 'Max retries exceeded', attempt: maxRetries };
}
}
// Setup routes
const router = new WebhookRouter();
router.addRoute('payment', 'http://order-service:3001/webhooks');
router.addRoute('customer', 'http://notification-service:3002/webhooks');
router.addRoute('github.push', 'http://ci-service:3003/webhooks');
router.addRoute('github', 'http://analytics-service:3004/webhooks');
Expected output: Router maps event type prefixes to internal consumer URLs. Multiple consumers can receive the same event. Each delivery has configurable retry settings.
Monitoring Dashboard
class RelayMonitor {
constructor() {
this.metrics = {
verifications: { success: 0, failed: 0 },
transformations: new Map(), // provider -> count
deliveries: { success: 0, failed: 0 },
errors: [],
recentDeliveries: [],
startTime: Date.now(),
};
}
recordVerification(provider, success) {
if (success) this.metrics.verifications.success++;
else this.metrics.verifications.failed++;
}
recordTransformation(provider, eventType) {
const key = `${provider}:${eventType}`;
this.metrics.transformations.set(
key,
(this.metrics.transformations.get(key) || 0) + 1
);
}
recordDelivery(provider, payload, results) {
const allSuccess = results.every(r => r.success);
if (allSuccess) this.metrics.deliveries.success++;
else this.metrics.deliveries.failed++;
this.metrics.recentDeliveries.push({
id: payload.id,
type: payload.type,
provider,
results,
timestamp: new Date().toISOString(),
});
if (this.metrics.recentDeliveries.length > 1000) {
this.metrics.recentDeliveries.shift();
}
}
recordError(provider, error) {
this.metrics.errors.push({
provider,
error: error.message,
timestamp: new Date().toISOString(),
});
if (this.metrics.errors.length > 100) {
this.metrics.errors.shift();
}
}
getSummary() {
const totalVerifications = this.metrics.verifications.success +
this.metrics.verifications.failed;
const totalDeliveries = this.metrics.deliveries.success +
this.metrics.deliveries.failed;
return {
uptime: Math.floor((Date.now() - this.metrics.startTime) / 1000),
verifications: {
total: totalVerifications,
success: this.metrics.verifications.success,
failed: this.metrics.verifications.failed,
successRate: totalVerifications > 0
? ((this.metrics.verifications.success / totalVerifications) * 100).toFixed(2)
: '100.00',
},
deliveries: {
total: totalDeliveries,
success: this.metrics.deliveries.success,
failed: this.metrics.deliveries.failed,
successRate: totalDeliveries > 0
? ((this.metrics.deliveries.success / totalDeliveries) * 100).toFixed(2)
: '100.00',
},
recentErrors: this.metrics.errors.slice(-5),
recentDeliveries: this.metrics.recentDeliveries.slice(-10),
};
}
}
// Monitoring endpoint
const monitor = new RelayMonitor();
relay.app.get('/relay/health', (req, res) => {
res.json(monitor.getSummary());
});
relay.app.get('/relay/metrics', (req, res) => {
// Prometheus-formatted metrics
const lines = [
'# HELP relay_verifications_total Total webhook verifications',
'# TYPE relay_verifications_total counter',
`relay_verifications_total{status="success"} ${monitor.metrics.verifications.success}`,
`relay_verifications_total{status="failed"} ${monitor.metrics.verifications.failed}`,
'# HELP relay_deliveries_total Total webhook deliveries',
'# TYPE relay_deliveries_total counter',
`relay_deliveries_total{status="success"} ${monitor.metrics.deliveries.success}`,
`relay_deliveries_total{status="failed"} ${monitor.metrics.deliveries.failed}`,
];
res.type('text/plain').send(lines.join('\n'));
});
Expected output: Monitoring dashboard provides real-time metrics: verification success rate, delivery success rate, recent errors, and recent deliveries. Prometheus metrics endpoint for Grafana integration.
Running the Relay
// Main entry point
const registry = new ProviderHandlerRegistry();
registry.register('stripe', new StripeHandler());
registry.register('github', new GitHubHandler());
// Setup routes to internal services
const router = new WebhookRouter();
router.addRoute('payment', 'http://localhost:4001/webhooks');
router.addRoute('customer', 'http://localhost:4002/webhooks');
router.addRoute('github', 'http://localhost:4003/webhooks');
// Start the relay
const relay = new WebhookRelay({
port: 3000,
registry,
router,
monitor,
});
relay.start();
// Provider sends webhooks to:
// Stripe: POST http://localhost:3000/relay/stripe
// GitHub: POST http://localhost:3000/relay/github
Expected output: Relay runs on port 3000. Providers send to /relay/:provider endpoints. The relay verifies, transforms, and routes to internal services. Health and metrics endpoints provide visibility.
Common Mistakes
1. No Provider Isolation
If one provider handler crashes, it affects all providers. Wrap each handler in try/catch. Use worker threads or separate processes for provider handlers if needed.
2. Hardcoded Consumer URLs
Consumer URLs should be configurable via environment variables or a service discovery system. Hardcoded URLs require code changes when services move.
3. Synchronous Routing
Routing to consumers synchronously delays the relay response. Route asynchronously where possible. Return the relay acknowledgment quickly. Process consumer deliveries in the background.
4. Missing Timeout on Consumer Delivery
A slow consumer blocks the relay for all other webhooks. Set timeouts on consumer delivery. Use circuit breakers for consistently slow consumers.
5. No Dead Letter Queue
If a consumer is permanently down, webhooks destined for it are stuck. Implement a dead letter queue. Notify operators. Provide a retry mechanism for dead-lettered webhooks.
Practice Questions
1. What is the purpose of the webhook relay pattern?
The relay decouples external providers from internal services. It handles provider-specific verification and transformation. Internal services receive standardized payloads. Adding new providers does not require changes to internal services.
2. How does the relay handle provider-specific verification?
The relay uses a provider handler registry. Each provider has a handler class that implements verify() with provider-specific logic. The relay delegates verification to the appropriate handler based on the URL path.
3. Why transform payloads to a standard format?
Internal services should not know about provider-specific payload formats. Transformation normalizes field names, date formats, and structure. Services process a consistent format regardless of the originating provider.
4. How do you add a new provider to the relay?
Create a new handler class implementing verify() and transform(). Register it with the registry. Create a new route in Express. No changes to the router, delivery service, or internal consumers.
Challenge
Extend the relay with: SendGrid provider handler, webhook transformation to CloudEvents format, circuit breaker for slow consumers, consumer registration API (dynamic route addition), webhook replay from dead letter queue, and a rate limiter per provider.
FAQ
Mini Project: Complete Relay Deployment
Deploy the webhook relay with: Express server on port 3000, provider handlers for Stripe and GitHub, transformation to Standard Webhooks format, routing to 3 internal services, delivery service with retries, monitoring and metrics endpoints, dead letter queue, Docker configuration for Containerization, and docker-compose for local development.
What's Next
Now that you have built a complete webhook system, explore Database Migration Strategies for managing schema changes in your webhook data storage.
Built by the developers of DodaTech
Doda Browser, DodaZIP & Durga Antivirus Pro