Synthetic Monitoring: Playwright and Lighthouse CI

In this tutorial, you'll learn about Synthetic Monitoring: Playwright and Lighthouse CI. We cover key concepts, practical examples, and best practices to help you understand and apply this topic effectively.

What You Will Learn

This tutorial teaches you how to implement synthetic monitoring using Playwright for automated browser testing and Lighthouse CI for performance auditing, enabling proactive detection of web application issues before users encounter them.

Why It Matters

Passive monitoring only tells you about problems after users are affected. Synthetic monitoring runs proactive checks against your application from controlled environments, catching regressions in functionality, performance, and availability before a single user sees an error.

Real-World Use

The DodaTech web team runs 25 synthetic monitors against the Doda Browser download page every 5 minutes from three geographic regions. When a CDN configuration error caused the download button to return a 502 error, the synthetic monitor detected the failure within 60 seconds and automatically rolled back the CDN change.

Synthetic monitoring uses scripted tests that simulate user interactions with your application. These tests run on a schedule from various locations, measuring response times, validating content, and checking functional behavior. Playwright provides the browser automation layer, while Lighthouse CI adds performance auditing.

Prerequisites

• Node.js 18+ installed
• A web application to monitor (your own or any public site)
• Docker installed for running tests in containers
• Basic knowledge of JavaScript

Step-by-Step Tutorial

Step 1: Set Up a Playwright Project

mkdir synthetic-monitoring && cd synthetic-monitoring
npm init -y
npm install @playwright/test
npx playwright install chromium

Step 2: Create Your First Synthetic Test

Create tests/homepage.spec.js:

const { test, expect } = require("@playwright/test");

test("Homepage loads successfully", async ({ page }) => {
    const start = Date.now();
    await page.goto("https://your-app.dodatech.com", {
        waitUntil: "networkidle"
    });
    const loadTime = Date.now() - start;

    // Assert page loaded
    await expect(page).toHaveTitle(/DodaTech/);

    // Report metrics
    console.log(JSON.stringify({
        test: "homepage-load",
        duration_ms: loadTime,
        status: "pass",
        timestamp: new Date().toISOString(),
    }));
});

Step 3: Add Functional Assertions

test("User can navigate to pricing page", async ({ page }) => {
    await page.goto("https://your-app.dodatech.com");
    await page.click("text=Pricing");
    await expect(page).toHaveURL(/\/pricing/);
    await expect(page.locator(".pricing-table")).toBeVisible();
});

test("Search returns results", async ({ page }) => {
    await page.goto("https://your-app.dodatech.com");
    await page.fill("[name='search']", "monitoring");
    await page.press("[name='search']", "Enter");
    await page.waitForSelector(".search-results");
    const results = await page.locator(".search-result").count();
    expect(results).toBeGreaterThan(0);
});

Step 4: Measure Performance Metrics

test("Measure Core Web Vitals", async ({ page }) => {
    await page.goto("https://your-app.dodatech.com", {
        waitUntil: "networkidle"
    });

    const metrics = await page.evaluate(() => ({
        fcp: performance.getEntriesByType("paint")
            .find(p => p.name === "first-contentful-paint")?.startTime,
        domContentLoaded: performance
            .getEntriesByType("navigation")[0]
            .domContentLoadedEventEnd,
        loadTime: performance
            .getEntriesByType("navigation")[0]
            .loadEventEnd,
    }));

    console.log(JSON.stringify(metrics));
    expect(metrics.fcp).toBeLessThan(2500);
});

Step 5: Run Tests on a Schedule

Create run-tests.sh:

#!/bin/bash
npx playwright test --reporter=json 2>/dev/null | \
  curl -X POST -H "Content-Type: application/json" \
  -d @- http://localhost:5000/api/synthetic-results

Add to crontab:

*/5 * * * * /path/to/run-tests.sh

Step 6: Set Up Lighthouse CI

npm install -g @lhci/cli

Create .lighthouserc.json:

{
    "ci": {
        "collect": {
            "url": [
                "https://your-app.dodatech.com",
                "https://your-app.dodatech.com/pricing",
                "https://your-app.dodatech.com/docs]
            ],
            "numberOfRuns": 3,
            "settings": {
                "throttlingMethod": "simulate",
                "formFactor": "mobile"
            }
        },
        "assert": {
            "preset": "lighthouse:no-pwa",
            "assertions": {
                "categories:performance": ["error", { "minScore": 0.9 }],
                "categories:accessibility": ["error", { "minScore": 0.9 }],
                "categories:best-practices": ["warn", { "minScore": 0.9 }],
                "categories:seo": ["error", { "minScore": 0.9 }]
            }
        },
        "upload": {
            "target": "filesystem",
            "outputDir": "./lhci-reports"
        }
    }
}

lhci autorun

Expected output: A Lighthouse report for each URL with scores and a summary of any assertion failures.

Step 7: Create a Synthetic Monitoring Receiver

from flask import Flask, request, jsonify
import json
import time

app = Flask(__name__)
results = []

@app.route("/api/synthetic-results", methods=["POST"])
def receive_results():
    data = request.get_data(as_text=True)
    for line in data.strip().split("\n"):
        try:
            result = json.loads(line)
            result["received_at"] = time.time()
            results.append(result)
        except json.JSONDecodeError:
            pass
    return jsonify({"received": len(data.splitlines())})

@app.route("/api/synthetic-results", methods=["GET"])
def get_results():
    return jsonify(results[-100:])

if __name__ == "__main__":
    app.run(port=5000)

Step 8: Expose Synthetic Metrics to Prometheus

from prometheus_client import Gauge, Histogram, start_http_server
import time

SYNTHETIC_DURATION = Histogram(
    "synthetic_test_duration_seconds",
    "Duration of synthetic tests",
    ["test_name", "status"]
)
SYNTHETIC_UP = Gauge(
    "synthetic_test_up",
    "Whether the synthetic test passed",
    ["test_name"]
)

def report_to_prometheus(result):
    test_name = result.get("test", "unknown")
    status = result.get("status", "unknown")
    duration = result.get("duration_ms", 0) / 1000
    SYNTHETIC_DURATION.labels(test_name=test_name, status=status).observe(duration)
    SYNTHETIC_UP.labels(test_name=test_name).set(1 if status == "pass" else 0)

Learning Path

flowchart LR
    A[Playwright Tests] --> B[Scheduled Execution]
    B --> C[Functional Validation]
    B --> D[Performance Metrics]
    A --> E[Lighthouse CI]
    E --> F[Performance Scores]
    E --> G[Accessibility Scores]
    C --> H[Results API]
    D --> H
    F --> H
    G --> H
    H --> I[Prometheus Metrics]
    H --> J[Alerting]
    style A fill:#4a90d9,color:#fff
    style H fill:#e67e22,color:#fff

Common Errors

1. Playwright test times out waiting for selector -- The element is not present due to a JavaScript error or the page changed. Increase the timeout and verify the selector is still valid.

2. Lighthouse CI fails with "url not found" -- The URL returns a 404 or the page redirects unexpectedly. Test the URL manually first.

3. Synthetic tests return false positives in CI -- The CI environment has different network conditions. Use throttling settings that match production.

4. Headless browser consumes too much memory -- Too many parallel tests run simultaneously. Limit concurrency with --workers=1 or use a smaller test matrix.

5. Test assertions fail intermittently -- The page has dynamic content or A/B testing variations. Use flexible assertions that handle expected variations.

6. Lighthouse performance score differs from RUM data -- Lighthouse runs in a controlled environment. It measures lab data, which differs from field data. Both are valuable.

7. Cron job does not run the test -- The PATH does not include npx or the project dependencies. Use absolute paths in the cron command.

Practice Questions

1. What is the difference between synthetic monitoring and RUM? Answer: Synthetic monitoring runs proactive tests from controlled environments. RUM captures data from real users in production.

2. How does Playwright simulate user interactions? Answer: Playwright uses browser automation APIs to click, type, navigate, and wait for elements, simulating real user behavior.

3. What metrics does Lighthouse CI measure? Answer: Performance (LCP, TBT, CLS), Accessibility, Best Practices, SEO, and PWA readiness.

4. How should synthetic test results be stored for analysis? Answer: Send results to a REST API that stores them in a time-series database for trend analysis and alerting.

5. Why run synthetic tests from multiple geographic locations? Answer: To detect regional CDN issues, DNS resolution differences, and latency variations that affect users in specific regions.

Challenge

Build a complete synthetic monitoring system for a three-page web application (homepage, search, checkout). Write Playwright tests that: validate all critical user flows, measure FCP and LCP for each page, verify that API calls return correct data, and check that error states display properly. Configure Lighthouse CI to audit all three pages with mobile and desktop presets, enforcing a performance score of 85+. Set up a cron job that runs the tests every 5 minutes and sends results to a collection API. Create a Prometheus exporter that exposes test duration, pass/fail status, and performance scores. Build a Grafana dashboard showing: test pass rate over time, page load time trend, Lighthouse score trend, and geographic breakdown. Set up alerts for any test that fails twice consecutively.

FAQ