SendGrid Complete Project: Build a Production Transactional Email System

In this tutorial, you will learn about SendGrid Complete Project: Build a Production Transactional Email System. We cover key concepts, practical examples, and best practices to help you master this topic.

This project combines all SendGrid concepts into a production-ready transactional email system — user lifecycle emails, batch reports, event processing, and deliverability monitoring.

What You'll Learn

How to architect a complete transactional email system with SendGrid, integrating signup flows, password resets, batch reports, event webhooks, suppression management, and deliverability dashboards.

Why It Matters

A well-architected email system is reliable, scalable, and maintainable. DodaTech's transactional email system processes 500K+ emails monthly with 99%+ deliverability, automated bounce handling, and real-time monitoring.

Real-World Use

A user signs up, receives a welcome email (template), opts into weekly reports (suppression group), receives reports (batch with personalizations), clicks a link (tracked), and the click is logged to the analytics database.

flowchart LR
    A["User Signs Up"] --> B["Send Welcome\nEmail via API"]
    B --> C["Create Contact\nAdd to List"]
    C --> D["Weekly Report\nBatch Personalizations"]
    D --> E["User Clicks\nLink in Email"]
    E --> F["Event Webhook\nLogs Click"]
    F --> G["Analytics\nDashboard"]
    D --> H["Bounce Event\nWebhook Handler"]
    H --> I["Auto-Remove\nfrom List"]
    style A fill:#dbeafe,stroke:#2563eb
    style B fill:#fef3c7,stroke:#d97706
    style F fill:#bbf7d0,stroke:#16a34a

Project Structure

email-system/
  app.py                  # Web app (Flask/FastAPI)
  send_email.py           # Send functions
  templates/              # Email templates
    welcome.html
    password_reset.html
    weekly_report.html
  webhooks.py             # Event + Parse webhook handlers
  utils.py                # Helper functions
  config.py               # Configuration
  requirements.txt

Welcome Email Flow

# config.py
import os

SENDGRID_API_KEY = os.environ["SENDGRID_API_KEY"]
FROM_EMAIL = "welcome@dodatech.com"
FROM_NAME = "DodaTech Security"
TEMPLATES = {
    "welcome": "d-welcome123",
    "password_reset": "d-pwreset456",
    "weekly_report": "d-report789",
}
SUPPRESSION_GROUPS = {
    "alerts": 1,
    "reports": 2,
    "promotions": 3,
}

# app.py
from flask import Flask, request, jsonify
from send_email import send_welcome, send_password_reset, send_batch_reports
from webhooks import handle_event_webhook, handle_inbound_parse

app = Flask(__name__)

@app.route("/api/signup", methods=["POST"])
def signup():
    data = request.get_json()
    user = create_user(data)

    send_welcome(user["email"], user["name"], user["tier"])
    log_event("welcome_sent", {"user_id": user["id"]})

    return jsonify({"status": "ok"}), 201

@app.route("/api/reset-password", methods=["POST"])
def reset_password():
    data = request.get_json()
    token = generate_reset_token(data["email"])

    send_password_reset(data["email"], data["name"], token)
    return jsonify({"status": "ok"}), 200

Batch Weekly Reports

# send_email.py
def send_batch_weekly_reports(users):
    message = Mail()
    message.from_email = Email("reports@dodatech.com", "DodaTech Security")
    message.template_id = TEMPLATES["weekly_report"]
    message.asm = Asm(GroupId(SUPPRESSION_GROUPS["reports"]))

    batch_size = 1000
    for i in range(0, len(users), batch_size):
        batch = users[i:i + batch_size]

        for user in batch:
            personalization = Personalization()
            personalization.add_to(Email(user["email"]))
            personalization.dynamic_template_data = {
                "name": user["name"],
                "threats": user["threats"],
                "report_url": f"https://dodatech.com/reports/{user['id']}",
                "scan_date": user["last_scan"]
            }
            personalization.add_custom_arg("user_id", user["id"])
            personalization.add_custom_arg("batch", str(i // batch_size))
            message.add_personalization(personalization)

        response = sg.send(message)
        print(f"Batch {i // batch_size}: Sent {len(batch)} reports ({response.status_code})")

        # Clear personalizations for next batch
        message.personalizations = []

Event Processing Queue

# webhooks.py
from flask import request, jsonify
import hashlib
import hmac

def verify_webhook(request):
    signature = request.headers.get("X-Twilio-Email-Event-Webhook-Signature")
    timestamp = request.headers.get("X-Twilio-Email-Event-Webhook-Timestamp")
    payload = timestamp + request.get_data(as_text=True)
    expected = hmac.new(
        os.environ["SENDGRID_WEBHOOK_SECRET"].encode(),
        payload.encode(),
        hashlib.sha256
    ).hexdigest()
    return hmac.compare_digest(signature, expected)

@app.route("/webhooks/sendgrid/events", methods=["POST"])
def events():
    if not verify_webhook(request):
        return jsonify({"error": "Invalid signature"}), 403

    events_data = request.get_json()
    for event in events_data:
        event_type = event.get("event")
        email = event.get("email")

        if event_type == "bounce" and event.get("bounce_type") == "permanent":
            remove_from_list(email)
        elif event_type == "click":
            record_clicks(email, event.get("url"))
        elif event_type == "open":
            record_open(email)
        elif event_type == "spam_report":
            flag_user(email)

    return jsonify({"status": "ok"}), 200

Deliverability Dashboard

# utils.py
from datetime import datetime, timedelta

def generate_deliverability_report(days=7):
    start = (datetime.now() - timedelta(days=days)).strftime("%Y-%m-%d")
    end = datetime.now().strftime("%Y-%m-%d")

    params = {
        "start_date": start,
        "end_date": end,
        "aggregated_by": "day",
        "categories": "weekly-report"
    }

    response = sg.client.stats.get(query_params=params)
    stats = response.to_dict()

    report = {}
    for day in stats:
        metrics = day.get("stats", [{}])[0].get("metrics", {})
        date = day["date"]
        report[date] = {
            "sent": metrics.get("requests", 0),
            "delivered": metrics.get("delivered", 0),
            "bounces": metrics.get("bounces", 0),
            "opens": metrics.get("unique_opens", 0),
            "clicks": metrics.get("unique_clicks", 0)
        }

    return report

Common Mistakes

1. No Separation Of Concerns

Mixing send logic, Webhook handling, and analytics in one file creates spaghetti. Separate into modules: sender, webhooks, templates, analytics.

2. Synchronous Webhook Processing

Processing events synchronously in the webhook handler causes timeouts. Push to a queue (Redis, Celery) and Process asynchronously.

3. Not Handling Retries

Network failures, rate limits, and API errors happen. Implement retry logic with exponential backoff for all API calls.

4. Missing Monitoring

Without monitoring (deliverability, bounce rates, webhook processing), you won't know when things break. Set up metrics and alerts.

5. Ignoring Testing

Test the entire flow: send test emails with different templates, verify webhooks process correctly, check bounce handling, and validate analytics data.

Practice Questions

1. How do you structure a production email system?
2. How do you handle batch sending with personalizations at scale?
3. What monitoring should a production email system have?
4. How do you test the complete email flow?

Answers:

1. Separate concerns into modules: send functions, webhook handlers, template management, analytics/monitoring. Use queues for async processing.
2. Split recipients into batches of 1000, create personalizations per batch, send, clear personalizations, repeat. Track batch indexes in custom arguments.
3. Monitor deliverability (delivery rate, bounce rate, spam rate, open/click rates), webhook processing (events per minute, error rate), and API usage (rate limits, failures).
4. Use test API keys and SendGrid's test mode. Send test emails to your own address. Simulate webhook events with sample payloads. Verify end-to-end: signup -> email -> webhook -> database.

Challenge: Build the complete email system: implement signup welcome flow with template, batch weekly reports with personalizations (1000 per batch), event webhook processing (delivered, open, click, bounce), automatic bounce handling, per-user suppression preferences, and a deliverability dashboard with daily metrics.

FAQ

Mini Project

Build and deploy the complete transactional email system: signup flow with welcome template, batch weekly reports (personalizations, 1000/batch), event webhook processing (verify, deduplicate, store), automatic bounce suppression, preference center with suppression groups, and a deliverability monitoring dashboard.

What's Next

You've completed the SendGrid learning path. Explore Twilio SMS API for SMS notifications as an alternative communication channel.