Skip to content

Python Requests: HTTP Client for API Test Automation

DodaTech Updated 2026-06-28 6 min read

In this tutorial, you will learn about Python Requests: HTTP Client for API Test Automation. We cover key concepts, practical examples, and best practices to help you master this topic.

The Python requests library is the de facto HTTP client for API testing, providing a clean API for GET/POST/PUT/DELETE requests, session management with cookies, authentication helpers, file uploads, and response validation.

What You'll Learn

How to use the requests library for API test automation: all HTTP methods, session objects for persistent connections, authentication (Basic, Bearer, API keys), file uploads and downloads, timeout and retry configuration, response validation, and error handling.

Why It Matters

Requests is the most popular Python HTTP library with 100M+ weekly downloads. Its simple, intuitive API makes it ideal for writing readable, maintainable API tests. DodaTech uses requests in all Python-based API test suites.

Real-World Use

A DodaTech QA engineer writes a test script that uses requests.Session to maintain authentication across API calls, sends GET/POST/PUT/DELETE requests to test a CRUD workflow, validates status codes and response bodies, and handles timeouts gracefully.

flowchart LR
    A["requests.\nSession()"] --> B["Session\nHeaders"]
    B --> C["GET /users\nList"]
    B --> D["POST /users\nCreate"]
    D --> E["GET /users/:id\nVerify"]
    E --> F["PUT /users/:id\nUpdate"]
    F --> G["DELETE /users/:id\nCleanup"]
    style A fill:#dbeafe,stroke:#2563eb
    style E fill:#bbf7d0,stroke:#16a34a
    style G fill:#fecaca,stroke:#dc2626

Basic Request Methods

import requests

BASE_URL = "https://api.dodatech.com/v1"

# GET request
response = requests.get(f"{BASE_URL}/users")
print(f"Status: {response.status_code}")
print(f"Users count: {len(response.json())}")
# Expected output:
# Status: 200
# Users count: 10

# POST request
new_user = {
    "email": "requests-test@example.com",
    "name": "Requests Test User",
    "password": "SecurePass123!"
}
response = requests.post(f"{BASE_URL}/users", json=new_user)
print(f"Status: {response.status_code}")
print(f"Created user ID: {response.json()['id']}")
# Expected output:
# Status: 201
# Created user ID: 42

# PUT request
update_data = {"name": "Updated Name"}
response = requests.put(f"{BASE_URL}/users/42", json=update_data)
print(f"Status: {response.status_code}")
print(f"Updated name: {response.json()['name']}")
# Expected output:
# Status: 200
# Updated name: Updated Name

# DELETE request
response = requests.delete(f"{BASE_URL}/users/42")
print(f"Status: {response.status_code}")
# Expected output:
# Status: 204

Session Management

import requests

# Session maintains cookies, headers, and connection pooling
session = requests.Session()

# Set default headers for all requests
session.headers.update({
    "Content-Type": "application/json",
    "Accept": "application/json",
    "User-Agent": "DodaTech-Test/1.0"
})

# Authenticate and store token
auth_response = session.post(f"{BASE_URL}/auth/login", json={
    "email": "test@dodatech.com",
    "password": "test-password"
})
token = auth_response.json()["token"]

# Set auth header for all subsequent requests
session.headers["Authorization"] = f"Bearer {token}"

# All requests now use the session's headers
response = session.get(f"{BASE_URL}/users/me")
print(f"My profile: {response.json()['email']}")
# Expected output:
# My profile: test@dodatech.com

# Connection pooling improves performance
# Session automatically reuses TCP connections

Authentication Methods

import requests
from requests.auth import HTTPBasicAuth, HTTPDigestAuth

# Basic Authentication
response = requests.get(
    "https://api.example.com/basic-auth",
    auth=HTTPBasicAuth("username", "password")
)
print(f"Basic auth: {response.status_code}")

# Bearer Token (JWT)
headers = {"Authorization": "Bearer eyJhbGciOiJIUzI1NiJ9..."}
response = requests.get("https://api.example.com/protected", headers=headers)

# API Key in Header
headers = {"X-API-Key": "sk_live_abc123def456"}
response = requests.get("https://api.example.com/data", headers=headers)

# API Key in Query String
response = requests.get(
    "https://api.example.com/data",
    params={"api_key": "sk_live_abc123def456"}
)

# Custom Auth Handler
class CustomTokenAuth(requests.auth.AuthBase):
    def __init__(self, token):
        self.token = token

    def __call__(self, request):
        request.headers["Authorization"] = f"Bearer {self.token}"
        request.headers["X-Request-Id"] = "test-request"
        return request

response = requests.get(
    "https://api.example.com/protected",
    auth=CustomTokenAuth("my-token")
)

File Uploads and Downloads

import requests

# File upload
with open("test-avatar.png", "rb") as f:
    response = requests.post(
        f"{BASE_URL}/upload/avatar",
        files={"file": ("avatar.png", f, "image/png")},
        data={"description": "User avatar upload test"}
    )
print(f"Upload status: {response.status_code}")
print(f"File URL: {response.json()['url']}")

# Upload with multiple files
files = [
    ("documents", ("report1.pdf", open("report1.pdf", "rb"), "application/pdf")),
    ("documents", ("report2.pdf", open("report2.pdf", "rb"), "application/pdf")),
]
response = requests.post(f"{BASE_URL}/upload/documents", files=files)
for f in files:
    f[1][1].close()

# File download with streaming
response = requests.get(
    f"{BASE_URL}/download/report.pdf",
    stream=True
)
if response.status_code == 200:
    with open("downloaded-report.pdf", "wb") as f:
        for chunk in response.iter_content(chunk_size=8192):
            f.write(chunk)
    print(f"Downloaded: {len(response.content)} bytes")

Timeouts and Retries

import requests
from requests.adapters import HTTPAdapter
from urllib3.util.retry import Retry

# Setting timeouts
try:
    response = requests.get(
        f"{BASE_URL}/slow-endpoint",
        timeout=(3.0, 10.0)  # (connect timeout, read timeout)
    )
except requests.Timeout:
    print("Request timed out")

# Automatic retries with backoff
session = requests.Session()
retry_strategy = Retry(
    total=3,                    # Max retries
    backoff_factor=1,           # Wait 1, 2, 4 seconds between retries
    status_forcelist=[429, 500, 502, 503, 504],  # Retry on these statuses
    allowed_methods=["GET", "POST", "PUT", "DELETE"]
)
adapter = HTTPAdapter(max_retries=retry_strategy)
session.mount("http://", adapter)
session.mount("https://", adapter)

# Requests using this session will auto-retry on failures
response = session.get(f"{BASE_URL}/users")
print(f"Response with retries: {response.status_code}")

Common Mistakes

1. Not Using Sessions for Multiple Requests

Each requests.get() creates a new TCP connection. Use requests.Session() for connection pooling, which reduces latency and server load for multi-request test flows.

2. Ignoring Timeouts

Without timeouts, tests hang indefinitely if the server doesn't respond. Always set timeouts: timeout=(3.0, 10.0) for connect and read timeouts respectively.

3. Not Checking response.ok

Always check the response status. Use response.raise_for_status() to automatically raise HTTPError for 4xx/5xx, or check response.ok before accessing .json().

4. Forgetting to Close File Handles

Uploaded files opened with open() stay open if not closed. Use context managers (with open() as f:) to ensure proper cleanup. For multiple files, close them after the request.

5. Overlooking Response Encoding

Requests auto-detects encoding but can guess wrong for some APIs. Explicitly set encoding: response.encoding = 'utf-8' or use response.content for bytes.

Practice Questions

  1. What is the advantage of using requests.Session?
  2. How do you handle authentication with different schemes?
  3. How do you upload files with additional form data?
  4. What is the difference between connect and read timeouts?

Answers:

  1. Session provides connection pooling (reuses TCP connections), persistent cookies, default headers, and auth configuration. This improves performance and simplifies multi-request test flows.
  2. Use auth=HTTPBasicAuth(user, pass) for Basic, headers={"Authorization": "Bearer <token>"} for Bearer, or create a custom AuthBase subclass for complex auth schemes.
  3. Use files={"field": ("filename", file_obj, "content-type")} for files and data={"field": "value"} for form fields in the same request. Requests handles multipart encoding automatically.
  4. Connect timeout is the time to establish the TCP connection. Read timeout is the time between bytes received. Set both: timeout=(3.0, 10.0) means 3s to connect, 10s between bytes.

Challenge: Build a complete API test client with requests: create a session with auth, implement CRUD operations for users and products, handle file upload/download, configure retries with exponential backoff for flaky endpoints, validate all responses with status code and schema checks, and test timeout handling.

FAQ

How do I send JSON data in a request?

Use the json parameter: requests.post(url, json={'key': 'value'}). This automatically sets Content-Type to application/json and serializes the dict to JSON.

How do I send form-encoded data?

Use the data parameter: requests.post(url, data={'key': 'value'}). This sends application/x-www-form-urlencoded. For file uploads, use the files parameter.

How do I access response headers?

response.headers returns a CaseInsensitiveDict. Access like: response.headers['Content-Type'] or response.headers.get('X-RateLimit-Remaining').

How do I handle redirects?

By default, requests follows redirects. Use allow_redirects=False to disable. Access response.history to see the redirect chain.

Can I make parallel requests with requests library?

Use concurrent.futures.ThreadPoolExecutor with requests. For async, use httpx or aiohttp instead. Requests is synchronous by design.

Mini Project

Build a Python API test suite using requests: session management with auth, CRUD operations for 3 resources, file upload/download, retry configuration for rate-limited endpoints, response validation helper functions, timeout testing, and parallel execution with ThreadPoolExecutor.

What's Next

REST Assured Java — test Java APIs with REST Assured.

← Previous Pytest for API Testing: Python Test Framework for REST Endpoints Next → REST Assured: Java API Testing Framework for RESTful Services

Built by the developers of DodaTech

Doda Browser, DodaZIP & Durga Antivirus Pro

Home Browse Api Automated Testing