API Test Automation Project: Build a Complete Testing Suite
In this tutorial, you will learn about API Test Automation Project: Build a Complete Testing Suite. We cover key concepts, practical examples, and best practices to help you master this topic.
Build a production-ready API test automation project combining test framework, Postman collections, Pytest integration tests, Pact contract tests, k6 load tests, CI/CD pipeline, production monitoring, and Allure reporting.
What You'll Learn
How to architect and implement a complete API test automation project combining functional tests (Postman + Pytest), contract tests (Pact), performance tests (k6), CI/CD integration (GitHub Actions), production monitoring (Prometheus + Grafana), and reporting (Allure).
Why It Matters
A comprehensive test automation Strategy covers all testing levels: functional correctness, contract Compliance, performance benchmarks, and production health. DodaTech's test automation pipeline runs 2,000+ tests across 5 stages in under 10 minutes.
Real-World Use
DodaTech deploys a new version of the payment API. The pipeline runs: Postman smoke tests (30s), Pytest integration tests with Stripe mock (90s), Pact contract verification (60s), k6 load test at 200 VUs (120s), and deploys if all pass. After deployment, Prometheus monitors latency and error rate.
flowchart LR
subgraph Development
A["Code\nCommit"] --> B["Postman\nSmoke Tests"]
end
subgraph CI_Pipeline
C["Pytest\nIntegration"]
D["Pact\nContract"]
E["k6 Load\nTest"]
end
subgraph Production
F["Prometheus\nMonitoring"]
G["Allure\nReports"]
end
B --> C
C --> D
D --> E
E -->|Pass| H["Deploy to\nProduction"]
H --> F
H --> G
style A fill:#dbeafe,stroke:#2563eb
style E fill:#6366f1,color:#fff
style H fill:#bbf7d0,stroke:#16a34a
style F fill:#fef3c7,stroke:#d97706
Project Structure
api-test-project/
postman/ # Postman collections
collections/
dodatech-smoke.json
dodatech-full.json
environments/
staging.json
production.json
data/
test-users.csv
ci/
run-smoke.sh
pytest-tests/ # Pytest integration tests
config/
settings.py
staging.yaml
core/
api_client.py
base_test.py
tests/
test_users.py
test_orders.py
test_payments.py
conftest.py
pytest.ini
requirements.txt
contract-tests/ # Pact contract tests
consumer/
order_service/
test_user_contract.py
provider/
user_service/
verify_contracts.py
pacts/
pact_broker.py
load-tests/ # k6 performance tests
scripts/
browse-flow.js
checkout-flow.js
spike-test.js
data/
products.json
thresholds.js
ci/
run-load-tests.sh
monitoring/ # Production monitoring
prometheus/
prometheus.yml
alerts.yml
grafana/
dashboards/
api-overview.json
api-latency.json
synthetic/
monitor.py
ci/ # CI/CD configuration
github-actions.yml
jenkinsfile.groovy
gitlab-ci.yml
reporting/ # Allure configuration
allure-config.yml
categories.json
environments/
requirements.txt
README.md
1. Postman Smoke Tests
// postman/collections/dodatech-smoke.json (simplified)
{
"info": {
"name": "DodaTech Smoke Tests",
"description": "Critical path smoke tests for every deployment"
},
"item": [
{
"name": "Health Check",
"event": [
{
"listen": "test",
"script": {
"exec": [
"pm.test('Health endpoint returns 200', function () {",
" pm.response.to.have.status(200);",
"});",
"pm.test('Response time < 500ms', function () {",
" pm.expect(pm.response.responseTime).to.be.below(500);",
"});"
]
}
}
],
"request": {
"method": "GET",
"url": "{{base_url}}/health"
}
}
]
}
# postman/ci/run-smoke.sh
#!/bin/bash
newman run collections/dodatech-smoke.json \
-e environments/staging.json \
--reporters cli,junit \
--reporter-junit-export results/smoke.xml \
--timeout-request 10000
echo "Smoke tests complete. Exit code: $?"
2. Pytest Integration Tests
# pytest-tests/tests/test_payments.py
import allure
import pytest
from core.base_test import BaseAPITest
@allure.epic("Payment Processing")
@allure.feature("Stripe Integration")
class TestPaymentAPI(BaseAPITest):
@allure.story("Create Payment")
@allure.severity(allure.severity_level.BLOCKER)
def test_create_payment_success(self):
with allure.step("Prepare payment data"):
payment_data = {
"order_id": "ORD-TEST-123",
"amount": 2999,
"currency": "usd",
"card_token": "tok_visa"
}
with allure.step("Send payment request"):
response = self.client.post("/payments", json=payment_data)
with allure.step("Verify payment created"):
self.assert_status(response, 201)
data = response.json()
assert data["status"] == "succeeded"
assert data["amount"] == 2999
@allure.story("Payment Validation")
@pytest.mark.parametrize("payload,expected_status", [
({"amount": -1}, 400),
({"amount": 0}, 400),
({"currency": "invalid"}, 400),
({}, 400),
])
def test_payment_validation(self, payload, expected_status):
response = self.client.post("/payments", json=payload)
self.assert_status(response, expected_status)
@allure.story("Payment Error Handling")
def test_payment_card_declined(self):
response = self.client.post("/payments", json={
"order_id": "ORD-DECLINED",
"amount": 5000,
"card_token": "tok_chargeDeclined"
})
self.assert_status(response, 402)
assert "declined" in response.json()["error"].lower()
3. k6 Load Tests
// load-tests/scripts/checkout-flow.js
import http from 'k6/http';
import { check, sleep } from 'k6';
import { Rate, Trend } from 'k6/metrics';
const checkoutDuration = new Trend('checkout_duration');
const errorRate = new Rate('checkout_errors');
export const options = {
stages: [
{ duration: '1m', target: 50 },
{ duration: '3m', target: 50 },
{ duration: '1m', target: 100 },
{ duration: '3m', target: 100 },
{ duration: '1m', target: 0 },
],
thresholds: {
http_req_duration: ['p(95)<1000', 'p(99)<2000'],
checkout_duration: ['p(95)<1500'],
checkout_errors: ['rate<0.02'],
},
};
const BASE_URL = __ENV.BASE_URL || 'https://api.staging.dodatech.com';
export default function () {
// Simulate checkout flow
const productResp = http.get(`${BASE_URL}/v1/products?limit=1`);
check(productResp, { 'products fetched': (r) => r.status === 200 });
if (productResp.json().length > 0) {
const product = productResp.json()[0];
sleep(1);
const cartResp = http.post(`${BASE_URL}/v1/cart`, JSON.stringify({
product_id: product.id,
quantity: 1,
}), { headers: { 'Content-Type': 'application/json' } });
check(cartResp, { 'item added to cart': (r) => r.status === 200 });
sleep(2);
const start = Date.now();
const checkoutResp = http.post(`${BASE_URL}/v1/checkout`, '{}');
checkoutDuration.add(Date.now() - start);
check(checkoutResp, { 'checkout completed': (r) => r.status === 200 });
if (checkoutResp.status !== 200) {
errorRate.add(1);
}
}
sleep(3);
}
4. CI/CD Pipeline
# .github/workflows/api-test-pipeline.yml
name: API Test Pipeline
on:
push:
branches: [main, develop]
pull_request:
branches: [main]
jobs:
smoke:
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@v4
- name: Smoke Tests
run: |
npm install -g newman
bash postman/ci/run-smoke.sh
- uses: actions/upload-artifact@v4
with:
name: smoke-results
path: postman/ci/results/
integration:
needs: smoke
runs-on: ubuntu-latest
services:
postgres:
image: postgres:16
env:
POSTGRES_DB: test
POSTGRES_USER: test
POSTGRES_PASSWORD: test
ports:
- 5432:5432
steps:
- uses: actions/checkout@v4
- uses: actions/setup-python@v5
with:
python-version: '3.12'
- run: |
pip install -r pytest-tests/requirements.txt
pytest pytest-tests/tests/ \
--alluredir=allure-results \
-n auto -v
- uses: actions/upload-artifact@v4
with:
name: allure-results
path: allure-results/
contract:
needs: integration
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@v4
- run: |
pip install pact-python
python contract-tests/provider/verify_contracts.py
load:
needs: contract
runs-on: ubuntu-latest
steps:
- uses: actions/setup-python@v5
- name: Load Tests
run: |
sudo apt-get install -y k6
k6 run load-tests/scripts/checkout-flow.js \
--out json=load-results.json
- uses: actions/upload-artifact@v4
with:
name: load-results
path: load-results.json
deploy:
needs: [smoke, integration, contract, load]
runs-on: ubuntu-latest
if: github.ref == 'refs/heads/main'
steps:
- run: echo "Deploying to production"
Common Mistakes
1. No Smoke Tests Before Full Suite
Running the full test suite (5 min) when the app is down (health check fails) wastes time. Always run smoke tests first (30s) to verify the app is reachable before running deeper tests.
2. Running Load Tests Without Baseline
Without baseline metrics, it's hard to know if 500ms is good or bad. Run load tests against a known-good version first to establish baselines for p50, p95, p99 latency and throughput.
3. Monitoring Without Synthetic Tests
Passive monitoring (APM, user traffic) misses issues when traffic is low. Add synthetic monitoring that probes every 30-60 seconds to ensure the API is functional even with zero user traffic.
4. No Trend Analysis in Reports
Single-run reports don't show regression direction. Allure history and Prometheus trends reveal if latency is creeping up, test count is dropping, or failure rate is increasing over time.
5. Ignoring Test Maintenance
Tests that aren't maintained become flaky and unreliable. Dedicate 20% of sprint capacity to test maintenance: update assertions for API changes, remove obsolete tests, and fix flaky tests.
Practice Questions
- What are the essential stages of a comprehensive API test pipeline?
- How do smoke tests differ from full test suites?
- How do you establish performance baselines?
- Why is test history important?
Answers:
- Smoke (quick health check) -> Integration (functional correctness) -> Contract (API compatibility) -> Load (performance) -> Deploy. Each stage gates the next. Smoke first for fast feedback on critical issues.
- Smoke tests check critical endpoints (health, login) in under 30 seconds. Full suites run detailed tests (all CRUD, error cases, edge cases) taking 5-10 minutes. Smoke gates the full suite.
- Run load tests against a known-good version (v1.0.0) and record p50, p95, p99 latency, throughput (req/s), and error rate at various VU levels. Compare new builds against these baselines. Fail if p95 degrades by more than 20%.
- Test history reveals trends: is latency increasing? Are more tests flaking? Is coverage dropping? A single pass/fail doesn't tell you if the system is getting better or worse over time.
Challenge: Build and deploy the complete API test automation project as described: set up all 5 components, create a working CI/CD pipeline with all stages, establish performance baselines with 3 k6 scenarios, configure Allure reporting with history, set up Prometheus + Grafana monitoring for production, and write a project README documenting the entire setup.
FAQ
Mini Project
The mini project is this complete API test automation project. Implement all 5 components, create a working CI/CD pipeline, establish performance baselines, configure Allure reporting with history and trends, set up production monitoring with Prometheus/Grafana, and document the entire system.
What's Next
OpenAPI Generator — generate API client libraries and server stubs from OpenAPI specs.
Built by the developers of DodaTech
Doda Browser, DodaZIP & Durga Antivirus Pro