Cloudflare Logpush -- Log Streaming to Destinations

In this tutorial, you'll learn about Cloudflare Logpush. We cover key concepts, practical examples, and best practices to help you understand and apply this topic effectively.

This tutorial explains how Cloudflare Logpush streams logs from the edge to your chosen destination, enabling long-term log storage, custom analytics, security auditing, and Compliance archiving without manual log collection.

Why Log Streaming Matters

Cloudflare processes billions of requests daily across its edge network. These requests generate logs containing HTTP headers, status codes, timing data, firewall actions, and cache status. Without Logpush, you can only view recent logs in the dashboard. For Compliance requirements, security investigations, or custom analytics, you need these logs in your own systems. Logpush delivers logs in near real-time to cloud storage (R2, S3, GCS), SIEM tools (Splunk, Datadog, Sumo Logic), or custom HTTP endpoints. The logs are structured as JSON lines, making them immediately usable with standard data processing tools.

Real-world use: Durga Antivirus Pro streams firewall logs to a private R2 bucket for Compliance auditing. The logs are ingested into a custom analytics pipeline that detects attack patterns across the entire user base. Logpush delivers over 10 million log entries per day with less than 3 minutes of delay from edge to bucket.

Logpush Data Flow

flowchart LR
  A[Request hits Cloudflare edge] --> B[Log generated]
  B --> C[Logpush aggregation]
  C --> D[Destination endpoint]
  D --> E[R2 bucket]
  D --> F[S3 compatible]
  D --> G[Datadog]
  D --> H[Splunk]
  D --> I[Custom HTTP]
  style B fill:#f90,color:#fff
  style C fill:#f90,color:#fff
  style D fill:#f90,color:#fff

Creating a Logpush Job

Logpush jobs are created via the Cloudflare API. You specify the destination, the dataset (HTTP requests, firewall events, DNS logs), and which fields to include.

# Create a Logpush job sending HTTP logs to R2
curl -X POST "https://api.cloudflare.com/client/v4/zones/ZONE_ID/logpush/jobs" \
  -H "Authorization: Bearer API_TOKEN" \
  -H "Content-Type: application/json" \
  --data '{
    "name": "HTTP Logs to R2",
    "destination_conf": "r2://bucket/logs/date={datetime}/account={account_id}",
    "dataset": "http_requests",
    "logpull_options": "fields=ClientIP,ClientRequestHost,ClientRequestMethod,EdgeResponseStatus,CacheCacheStatus,EdgeStartTimestamp",
    "enabled": true
  }'
# Expected output:
# {"success":true,"result":{"id":12345,"name":"HTTP Logs to R2","enabled":true}}

The destination URI uses a scheme prefix (r2://, s3://, <a href="/devops/monitoring-tools/">Datadog</a>://, splunk://) followed by destination-specific details. The logpull_options parameter controls which fields are included, reducing log volume and cost.

Available Datasets and Fields

Logpush supports multiple datasets. Each dataset contains different fields relevant to its domain.

# List all available datasets for a zone
curl -s "https://api.cloudflare.com/client/v4/zones/ZONE_ID/logpush/datasets" \
  -H "Authorization: Bearer API_TOKEN" | jq '.result[] | .name'
# Expected output:
# "http_requests"
# "firewall_events"
# "spectrum_events"
# "nel_reports"
# "dns_logs"
# "audit_logs"
# "workers_trace_events"

# Get available fields for HTTP requests dataset
curl -s "https://api.cloudflare.com/client/v4/zones/ZONE_ID/logpush/datasets/http_requests/fields" \
  -H "Authorization: Bearer API_TOKEN" | jq 'keys'
# Expected output:
# ["CacheCacheStatus","ClientIP","ClientRequestBytes","ClientRequestHost", "#  "ClientRequestMethod"","EdgeResponseStatus","OriginResponseTime","ZoneName"]

Selecting only the fields you need reduces log volume and destination storage costs. For security auditing, focus on ClientIP, ClientRequestHost, EdgeResponseStatus, and FirewallMatchesActions. For performance analysis, include EdgeStartTimestamp, OriginResponseTime, and CacheCacheStatus.

Configuring Destinations

Each destination type requires specific configuration parameters. Here are examples for S3-compatible storage and Datadog.

# Logpush to S3-compatible storage (Backblaze, MinIO)
curl -X POST "https://api.cloudflare.com/client/v4/zones/ZONE_ID/logpush/jobs" \
  -H "Authorization: Bearer API_TOKEN" \
  -H "Content-Type: application/json" \
  --data '{
    "name": "Firewall Events to S3",
    "destination_conf": "s3://my-bucket/logs?region=us-east-1&endpoint=https://s3.example.com",
    "dataset": "firewall_events",
    "ownership_challenge": "eyJ...challenge_token...",
    "enabled": true
  }'
# Expected output:
# {"success":true,"result":{"id":12346,"name":"Firewall Events to S3"}}

# Logpush to Datadog
curl -X POST "https://api.cloudflare.com/client/v4/zones/ZONE_ID/logpush/jobs" \
  -H "Authorization: Bearer API_TOKEN" \
  -H "Content-Type: application/json" \
  --data '{
    "name": "HTTP Logs to Datadog",
    "destination_conf": "datadog://http-intake.logs.datadoghq.com/v1/input?service=cloudflare&ddsource=cloudflare&host=example.com",
    "dataset": "http_requests",
    "enabled": true
  }'
# Expected output:
# {"success":true,"result":{"id":12347,"name":"HTTP Logs to <a href="/devops/monitoring-tools/">Datadog</a>"}}

For S3 destinations, you must complete an ownership challenge before logs start flowing. This involves creating a file in the bucket with a token provided by Cloudflare, proving you control the destination.

Monitoring Logpush Health

You can check the status of your Logpush jobs to ensure logs are flowing correctly. Common issues include destination permissions, network connectivity, and field selection errors.

# Check Logpush job status
curl -s "https://api.cloudflare.com/client/v4/zones/ZONE_ID/logpush/jobs/12345" \
  -H "Authorization: Bearer API_TOKEN" | jq '.result | {name, enabled, last_complete, error_message}'
# Expected output:
# {
#   "name": "HTTP Logs to R2",
#   "enabled": true,
#   "last_complete": "2026-06-23T12:00:00Z",
#   "error_message": null
# }

# List all jobs with their status
curl -s "https://api.cloudflare.com/client/v4/zones/ZONE_ID/logpush/jobs" \
  -H "Authorization: Bearer API_TOKEN" | jq '.result[] | {id, name, enabled, last_complete}'
# Expected output:
# {"id":12345,"name":"HTTP Logs to R2","enabled":true,"last_complete":"2026-06-23T12:00:00Z"}
# {"id":12346,"name":"Firewall Events to S3","enabled":true,"last_complete":"2026-06-23T11:55:00Z"}

If last_complete is far behind the current time or error_message is not null, investigate destination permissions, network access, or field configuration. Logpush retries failed deliveries for up to 72 hours before permanent failure.

FAQ

Practice Questions

1. What is the purpose of the ownership challenge when setting up an S3 Logpush destination?
2. Which dataset would you use to stream firewall-related events to a SIEM tool?
3. How do you verify that a Logpush job is delivering logs successfully?

Summary

Cloudflare Logpush streams edge logs to your chosen destinations in near real-time with JSON-formatted data. It supports multiple datasets including HTTP requests, firewall events, DNS logs, and Workers traces. Destinations include R2, S3-compatible storage, Datadog, Splunk, Sumo Logic, and custom HTTP endpoints. Field selection controls log volume and cost, while job status monitoring ensures reliable delivery. Logpush replaces manual log collection for Compliance, security auditing, and custom analytics use cases.

Built by the developers of Doda Browser, DodaZIP, and Durga Antivirus Pro -- security-first tools for the modern web.