Skip to content

Python httpx and aiohttp: Modern HTTP Clients for API Testing

DodaTech Updated 2026-06-28 6 min read

In this tutorial, you will learn about Python httpx and aiohttp: Modern HTTP Clients for API Testing. We cover key concepts, practical examples, and best practices to help you master this topic.

Modern Python HTTP clients like httpx and aiohttp provide async/await support, HTTP/2, connection pooling, streaming, and advanced features beyond the requests library, essential for testing modern async APIs.

What You'll Learn

How to use httpx (sync and async) and aiohttp for API testing, make concurrent requests with async/await, use connection pooling for performance, test Websocket endpoints, handle streaming responses, and choose between httpx and aiohttp.

Why It Matters

Modern APIs increasingly use async patterns, HTTP/2, WebSockets, and streaming. Requests library is synchronous only. httpx and aiohttp support these modern patterns. DodaTech uses httpx for async microservice tests and load testing.

Real-World Use

A DodaTech service calls 3 internal APIs in parallel. Using httpx async client, the test fires 3 concurrent requests, awaits all responses, validates each, and completes in the time of the slowest request (450ms) instead of 3 sequential calls (1.4s).

flowchart LR
    A["async\nClient"] --> B["Request 1\nGET /users"]
    A --> C["Request 2\nGET /products"]
    A --> D["Request 3\nGET /orders"]
    B --> E["await\nasyncio.gather()"]
    C --> E
    D --> E
    E --> F["All Responses\nReady"]
    F --> G["Assertions"]
    style A fill:#dbeafe,stroke:#2563eb
    style E fill:#bbf7d0,stroke:#16a34a

httpx Basic Usage

import httpx
import pytest

# Sync client
def test_get_users_sync():
    with httpx.Client(base_url="https://api.dodatech.com/v1") as client:
        response = client.get("/users")
        assert response.status_code == 200
        data = response.json()
        assert isinstance(data, list)
        assert len(data) > 0
        print(f"Users: {len(data)}")

# Async client
@pytest.mark.asyncio
async def test_get_users_async():
    async with httpx.AsyncClient(base_url="https://api.dodatech.com/v1") as client:
        response = await client.get("/users")
        assert response.status_code == 200
        data = response.json()
        assert isinstance(data, list)

Concurrent Requests with httpx

import asyncio
import httpx

async def fetch_user_data(user_id):
    async with httpx.AsyncClient() as client:
        user_resp = await client.get(
            f"https://api.dodatech.com/v1/users/{user_id}"
        )
        orders_resp = await client.get(
            f"https://api.dodatech.com/v1/users/{user_id}/orders"
        )
        return {
            "user": user_resp.json(),
            "orders": orders_resp.json()
        }

async def fetch_multiple_users(user_ids):
    async with asyncio.TaskGroup() as tg:
        tasks = [tg.create_task(fetch_user_data(uid)) for uid in user_ids]

    results = [task.result() for task in tasks]
    print(f"Fetched {len(results)} users with orders")
    for r in results:
        print(f"  {r['user']['email']}: {len(r['orders'])} orders")
    return results

# Run
# results = asyncio.run(fetch_multiple_users([1, 2, 3, 4, 5]))
# Expected output:
# Fetched 5 users with orders
#   user1@test.com: 3 orders
#   user2@test.com: 1 orders
#   user3@test.com: 0 orders

httpx Features for Testing

import httpx

# HTTP/2 support
client = httpx.Client(http2=True)
response = client.get("https://api.dodatech.com/v1/users")
print(f"HTTP version: {response.http_version}")
# Expected: HTTP/2

# Connection pooling
# httpx reuses connections automatically
# Configure pool size
limits = httpx.Limits(
    max_connections=100,
    max_keepalive_connections=20,
    keepalive_expiry=30.0
)
client = httpx.Client(limits=limits)

# Event hooks for monitoring
def log_request(request):
    print(f"Request: {request.method} {request.url}")

def log_response(response):
    print(f"Response: {response.status_code} ({response.elapsed.total_seconds():.2f}s)")

client = httpx.Client(
    event_hooks={
        "request": [log_request],
        "response": [log_response]
    }
)

# Mock transport for tests
import httpx

def test_with_mock():
    """Use mock transport to avoid real HTTP calls."""
    def handler(request):
        return httpx.Response(
            status_code=200,
            json={"id": 1, "email": "test@test.com"}
        )

    client = httpx.Client(transport=httpx.MockTransport(handler))
    response = client.get("https://api.test.com/users/1")
    assert response.json()["email"] == "test@test.com"

aiohttp for API Testing

import aiohttp
import asyncio
import pytest

# Basic aiohttp test
@pytest.mark.asyncio
async def test_with_aiohttp():
    async with aiohttp.ClientSession(
        base_url="https://api.dodatech.com/v1"
    ) as session:
        async with session.get("/users") as response:
            assert response.status == 200
            data = await response.json()
            assert len(data) > 0

# Concurrent requests with aiohttp
async def fetch_all_pages(session, base_url, total_pages):
    async def fetch_page(page):
        async with session.get(f"{base_url}/users?page={page}") as resp:
            return await resp.json()

    tasks = [fetch_page(p) for p in range(1, total_pages + 1)]
    results = await asyncio.gather(*tasks)
    all_users = [user for page in results for user in page]
    print(f"Total users across {total_pages} pages: {len(all_users)}")
    return all_users

# async with aiohttp.ClientSession() as session:
#     users = await fetch_all_pages(session, "https://api.dodatech.com/v1", 5)

WebSocket Testing

import asyncio
import httpx

# httpx WebSocket support
async def test_websocket():
    async with httpx.AsyncClient() as client:
        async with client.ws_connect(
            "wss://api.dodatech.com/v1/ws/notifications",
            headers={"Authorization": "Bearer token"}
        ) as websocket:
            # Send message
            await websocket.send_json({"type": "subscribe", "channel": "orders"})

            # Receive messages
            for _ in range(3):
                message = await websocket.receive_json()
                print(f"WS message: {message['type']}")
                assert message["status"] == "ok"

# aiohttp WebSocket
async def test_websocket_aiohttp():
    async with aiohttp.ClientSession() as session:
        async with session.ws_connect(
            "wss://api.dodatech.com/v1/ws",
            headers={"Authorization": "Bearer token"}
        ) as ws:
            await ws.send_json({"action": "ping"})
            async for msg in ws:
                if msg.type == aiohttp.WSMsgType.TEXT:
                    data = msg.json()
                    print(f"Received: {data}")
                    if data.get("type") == "pong":
                        break

Common Mistakes

1. Not Using Async Context Managers

async with httpx.AsyncClient() as client: properly closes connections. Forgetting async with leaks connections and causes test warnings about unclosed transports.

2. Mixing Sync and Async Without Care

Calling async code from sync tests requires asyncio.run() or pytest-asyncio. Mixing them without proper event loop management causes RuntimeError about conflicting event loops.

3. Not Setting Limits for Load Tests

Default connection limits are conservative. For load tests, explicitly set Limits(max_connections=1000) to allow concurrent requests without Connection Pool exhaustion.

4. Forgetting Response Body Await

In aiohttp, response.json() is a Coroutine. Forgetting await returns a coroutine object instead of data. In httpx, response.json() is synchronous.

5. Not Handling WebSocket Disconnects

WebSocket connections can drop. Always handle websocket.close() or ConnectionClosed exceptions. Implement reconnect logic for tests that need continuous WS connectivity.

Practice Questions

  1. What is the difference between httpx and requests?
  2. How do you make concurrent HTTP requests with httpx?
  3. How do you test WebSocket endpoints?
  4. What are httpx event hooks used for?

Answers:

  1. httpx supports async/await, HTTP/2, WebSockets, and modern connection pooling. requests is synchronous only, HTTP/1.1, with a simpler API. Use httpx for new projects, requests for legacy code.
  2. Use asyncio.gather(*tasks) with multiple async HTTP calls. Create tasks with client.get() inside async functions, gather them, and await all results concurrently.
  3. Use httpx: client.ws_connect(url) returns a WebSocket session with send_json() and receive_json(). Use aiohttp: session.ws_connect(url) with send_json() and async iteration over incoming messages.
  4. Event hooks allow custom callbacks on request/response lifecycle events. Use them for logging, timing, modifying requests, or injecting delays for testing.

Challenge: Build a test suite that uses async httpx for an e-commerce API: concurrent user + orders + products fetching, WebSocket notification testing, mock transport for offline testing, event hooks for latency measurement, and compare sync vs async performance.

FAQ

Should I use httpx or aiohttp for API testing?

httpx has a cleaner API, supports HTTP/2 and WebSockets, and has a sync mode too. aiohttp is more mature for WebSocket-heavy use cases. Prefer httpx unless you need aiohttp-specific features.

How do I handle redirects in httpx?

By default, httpx follows redirects. Set follow_redirects=False to disable. Access response.history for the redirect chain, response.next_request for the next request in the chain.

Can I use httpx with pytest fixtures?

Yes, create async fixtures with @pytest_asyncio.fixture. Yield the client in fixture body. Cleanup happens after the fixture's yield in async context.

How do I stream large responses?

Use client.stream(method, url) as an async context manager. Read response in chunks with response.aiter_bytes() or response.aiter_lines() for memory-efficient processing.

What is client-side caching in httpx?

httpx does not cache by default. Use Hishel (HTTP cache library) with httpx for client-side caching in tests to avoid repeated calls to the same endpoints.

Mini Project

Build an async API test suite using httpx: async CRUD tests for 3 resources, concurrent data fetching with asyncio.gather, WebSocket notification testing, mock transport for offline mode, event hooks for request logging, performance comparison with sync requests, and JUnit XML reporting.

What's Next

Mock Servers — mock external API dependencies for isolated testing.

Built by the developers of DodaTech

Doda Browser, DodaZIP & Durga Antivirus Pro