Skip to content

API Test Automation Project: Build a Complete Testing Suite

DodaTech Updated 2026-06-28 7 min read

In this tutorial, you will learn about API Test Automation Project: Build a Complete Testing Suite. We cover key concepts, practical examples, and best practices to help you master this topic.

Build a production-ready API test automation project combining test framework, Postman collections, Pytest integration tests, Pact contract tests, k6 load tests, CI/CD pipeline, production monitoring, and Allure reporting.

What You'll Learn

How to architect and implement a complete API test automation project combining functional tests (Postman + Pytest), contract tests (Pact), performance tests (k6), CI/CD integration (GitHub Actions), production monitoring (Prometheus + Grafana), and reporting (Allure).

Why It Matters

A comprehensive test automation Strategy covers all testing levels: functional correctness, contract Compliance, performance benchmarks, and production health. DodaTech's test automation pipeline runs 2,000+ tests across 5 stages in under 10 minutes.

Real-World Use

DodaTech deploys a new version of the payment API. The pipeline runs: Postman smoke tests (30s), Pytest integration tests with Stripe mock (90s), Pact contract verification (60s), k6 load test at 200 VUs (120s), and deploys if all pass. After deployment, Prometheus monitors latency and error rate.

flowchart LR
    subgraph Development
        A["Code\nCommit"] --> B["Postman\nSmoke Tests"]
    end
    subgraph CI_Pipeline
        C["Pytest\nIntegration"]
        D["Pact\nContract"]
        E["k6 Load\nTest"]
    end
    subgraph Production
        F["Prometheus\nMonitoring"]
        G["Allure\nReports"]
    end
    B --> C
    C --> D
    D --> E
    E -->|Pass| H["Deploy to\nProduction"]
    H --> F
    H --> G
    style A fill:#dbeafe,stroke:#2563eb
    style E fill:#6366f1,color:#fff
    style H fill:#bbf7d0,stroke:#16a34a
    style F fill:#fef3c7,stroke:#d97706

Project Structure

api-test-project/
  postman/                   # Postman collections
    collections/
      dodatech-smoke.json
      dodatech-full.json
    environments/
      staging.json
      production.json
    data/
      test-users.csv
    ci/
      run-smoke.sh

  pytest-tests/              # Pytest integration tests
    config/
      settings.py
      staging.yaml
    core/
      api_client.py
      base_test.py
    tests/
      test_users.py
      test_orders.py
      test_payments.py
    conftest.py
    pytest.ini
    requirements.txt

  contract-tests/            # Pact contract tests
    consumer/
      order_service/
        test_user_contract.py
    provider/
      user_service/
        verify_contracts.py
    pacts/
    pact_broker.py

  load-tests/                # k6 performance tests
    scripts/
      browse-flow.js
      checkout-flow.js
      spike-test.js
    data/
      products.json
    thresholds.js
    ci/
      run-load-tests.sh

  monitoring/                # Production monitoring
    prometheus/
      prometheus.yml
      alerts.yml
    grafana/
      dashboards/
        api-overview.json
        api-latency.json
    synthetic/
      monitor.py

  ci/                        # CI/CD configuration
    github-actions.yml
    jenkinsfile.groovy
    gitlab-ci.yml

  reporting/                 # Allure configuration
    allure-config.yml
    categories.json
    environments/

  requirements.txt
  README.md

1. Postman Smoke Tests

// postman/collections/dodatech-smoke.json (simplified)
{
    "info": {
        "name": "DodaTech Smoke Tests",
        "description": "Critical path smoke tests for every deployment"
    },
    "item": [
        {
            "name": "Health Check",
            "event": [
                {
                    "listen": "test",
                    "script": {
                        "exec": [
                            "pm.test('Health endpoint returns 200', function () {",
                            "    pm.response.to.have.status(200);",
                            "});",
                            "pm.test('Response time < 500ms', function () {",
                            "    pm.expect(pm.response.responseTime).to.be.below(500);",
                            "});"
                        ]
                    }
                }
            ],
            "request": {
                "method": "GET",
                "url": "{{base_url}}/health"
            }
        }
    ]
}
# postman/ci/run-smoke.sh
#!/bin/bash
newman run collections/dodatech-smoke.json \
  -e environments/staging.json \
  --reporters cli,junit \
  --reporter-junit-export results/smoke.xml \
  --timeout-request 10000
echo "Smoke tests complete. Exit code: $?"

2. Pytest Integration Tests

# pytest-tests/tests/test_payments.py
import allure
import pytest
from core.base_test import BaseAPITest

@allure.epic("Payment Processing")
@allure.feature("Stripe Integration")
class TestPaymentAPI(BaseAPITest):

    @allure.story("Create Payment")
    @allure.severity(allure.severity_level.BLOCKER)
    def test_create_payment_success(self):
        with allure.step("Prepare payment data"):
            payment_data = {
                "order_id": "ORD-TEST-123",
                "amount": 2999,
                "currency": "usd",
                "card_token": "tok_visa"
            }

        with allure.step("Send payment request"):
            response = self.client.post("/payments", json=payment_data)

        with allure.step("Verify payment created"):
            self.assert_status(response, 201)
            data = response.json()
            assert data["status"] == "succeeded"
            assert data["amount"] == 2999

    @allure.story("Payment Validation")
    @pytest.mark.parametrize("payload,expected_status", [
        ({"amount": -1}, 400),
        ({"amount": 0}, 400),
        ({"currency": "invalid"}, 400),
        ({}, 400),
    ])
    def test_payment_validation(self, payload, expected_status):
        response = self.client.post("/payments", json=payload)
        self.assert_status(response, expected_status)

    @allure.story("Payment Error Handling")
    def test_payment_card_declined(self):
        response = self.client.post("/payments", json={
            "order_id": "ORD-DECLINED",
            "amount": 5000,
            "card_token": "tok_chargeDeclined"
        })
        self.assert_status(response, 402)
        assert "declined" in response.json()["error"].lower()

3. k6 Load Tests

// load-tests/scripts/checkout-flow.js
import http from 'k6/http';
import { check, sleep } from 'k6';
import { Rate, Trend } from 'k6/metrics';

const checkoutDuration = new Trend('checkout_duration');
const errorRate = new Rate('checkout_errors');

export const options = {
    stages: [
        { duration: '1m', target: 50 },
        { duration: '3m', target: 50 },
        { duration: '1m', target: 100 },
        { duration: '3m', target: 100 },
        { duration: '1m', target: 0 },
    ],
    thresholds: {
        http_req_duration: ['p(95)<1000', 'p(99)<2000'],
        checkout_duration: ['p(95)<1500'],
        checkout_errors: ['rate<0.02'],
    },
};

const BASE_URL = __ENV.BASE_URL || 'https://api.staging.dodatech.com';

export default function () {
    // Simulate checkout flow
    const productResp = http.get(`${BASE_URL}/v1/products?limit=1`);
    check(productResp, { 'products fetched': (r) => r.status === 200 });

    if (productResp.json().length > 0) {
        const product = productResp.json()[0];
        sleep(1);

        const cartResp = http.post(`${BASE_URL}/v1/cart`, JSON.stringify({
            product_id: product.id,
            quantity: 1,
        }), { headers: { 'Content-Type': 'application/json' } });
        check(cartResp, { 'item added to cart': (r) => r.status === 200 });

        sleep(2);

        const start = Date.now();
        const checkoutResp = http.post(`${BASE_URL}/v1/checkout`, '{}');
        checkoutDuration.add(Date.now() - start);
        check(checkoutResp, { 'checkout completed': (r) => r.status === 200 });

        if (checkoutResp.status !== 200) {
            errorRate.add(1);
        }
    }

    sleep(3);
}

4. CI/CD Pipeline

# .github/workflows/api-test-pipeline.yml
name: API Test Pipeline
on:
  push:
    branches: [main, develop]
  pull_request:
    branches: [main]

jobs:
  smoke:
    runs-on: ubuntu-latest
    steps:
      - uses: actions/checkout@v4
      - name: Smoke Tests
        run: |
          npm install -g newman
          bash postman/ci/run-smoke.sh
      - uses: actions/upload-artifact@v4
        with:
          name: smoke-results
          path: postman/ci/results/

  integration:
    needs: smoke
    runs-on: ubuntu-latest
    services:
      postgres:
        image: postgres:16
        env:
          POSTGRES_DB: test
          POSTGRES_USER: test
          POSTGRES_PASSWORD: test
        ports:
          - 5432:5432
    steps:
      - uses: actions/checkout@v4
      - uses: actions/setup-python@v5
        with:
          python-version: '3.12'
      - run: |
          pip install -r pytest-tests/requirements.txt
          pytest pytest-tests/tests/ \
            --alluredir=allure-results \
            -n auto -v
      - uses: actions/upload-artifact@v4
        with:
          name: allure-results
          path: allure-results/

  contract:
    needs: integration
    runs-on: ubuntu-latest
    steps:
      - uses: actions/checkout@v4
      - run: |
          pip install pact-python
          python contract-tests/provider/verify_contracts.py

  load:
    needs: contract
    runs-on: ubuntu-latest
    steps:
      - uses: actions/setup-python@v5
      - name: Load Tests
        run: |
          sudo apt-get install -y k6
          k6 run load-tests/scripts/checkout-flow.js \
            --out json=load-results.json
      - uses: actions/upload-artifact@v4
        with:
          name: load-results
          path: load-results.json

  deploy:
    needs: [smoke, integration, contract, load]
    runs-on: ubuntu-latest
    if: github.ref == 'refs/heads/main'
    steps:
      - run: echo "Deploying to production"

Common Mistakes

1. No Smoke Tests Before Full Suite

Running the full test suite (5 min) when the app is down (health check fails) wastes time. Always run smoke tests first (30s) to verify the app is reachable before running deeper tests.

2. Running Load Tests Without Baseline

Without baseline metrics, it's hard to know if 500ms is good or bad. Run load tests against a known-good version first to establish baselines for p50, p95, p99 latency and throughput.

3. Monitoring Without Synthetic Tests

Passive monitoring (APM, user traffic) misses issues when traffic is low. Add synthetic monitoring that probes every 30-60 seconds to ensure the API is functional even with zero user traffic.

4. No Trend Analysis in Reports

Single-run reports don't show regression direction. Allure history and Prometheus trends reveal if latency is creeping up, test count is dropping, or failure rate is increasing over time.

5. Ignoring Test Maintenance

Tests that aren't maintained become flaky and unreliable. Dedicate 20% of sprint capacity to test maintenance: update assertions for API changes, remove obsolete tests, and fix flaky tests.

Practice Questions

  1. What are the essential stages of a comprehensive API test pipeline?
  2. How do smoke tests differ from full test suites?
  3. How do you establish performance baselines?
  4. Why is test history important?

Answers:

  1. Smoke (quick health check) -> Integration (functional correctness) -> Contract (API compatibility) -> Load (performance) -> Deploy. Each stage gates the next. Smoke first for fast feedback on critical issues.
  2. Smoke tests check critical endpoints (health, login) in under 30 seconds. Full suites run detailed tests (all CRUD, error cases, edge cases) taking 5-10 minutes. Smoke gates the full suite.
  3. Run load tests against a known-good version (v1.0.0) and record p50, p95, p99 latency, throughput (req/s), and error rate at various VU levels. Compare new builds against these baselines. Fail if p95 degrades by more than 20%.
  4. Test history reveals trends: is latency increasing? Are more tests flaking? Is coverage dropping? A single pass/fail doesn't tell you if the system is getting better or worse over time.

Challenge: Build and deploy the complete API test automation project as described: set up all 5 components, create a working CI/CD pipeline with all stages, establish performance baselines with 3 k6 scenarios, configure Allure reporting with history, set up Prometheus + Grafana monitoring for production, and write a project README documenting the entire setup.

FAQ

How long does a complete API test pipeline take?

Target under 10 minutes: smoke (30s) + integration (3min) + contract (1min) + load (4min) + deployment (1min). Optimize slow stages with parallel execution and test splitting.

How often should I run load tests?

Run load tests on every production deployment (not every commit). Run full load suite nightly. Quick smoke load test (1 min, 50 VUs) can run on every PR.

What metrics should I track across the project?

Track: test count trend, pass rate, flaky test count, execution time, code coverage, p95 latency, error rate, synthetic monitor uptime, and time-to-detection (how fast monitoring catches issues).

How do I handle test data across the entire project?

Use the same data factories across Postman (via data files), Pytest (via factory_boy), and k6 (via SharedArray). Centralize test data generation in a shared module.

What is the biggest risk in API test automation?

Test drift — tests pass but the real API has changed behavior. Combat drift with contract testing (Pact), schema validation in every test, and regular test audits to verify tests still test real behavior.

Mini Project

The mini project is this complete API test automation project. Implement all 5 components, create a working CI/CD pipeline, establish performance baselines, configure Allure reporting with history and trends, set up production monitoring with Prometheus/Grafana, and document the entire system.

What's Next

OpenAPI Generator — generate API client libraries and server stubs from OpenAPI specs.

Built by the developers of DodaTech

Doda Browser, DodaZIP & Durga Antivirus Pro