gRPC Benchmarking Tools — Measuring and Comparing gRPC Performance
In this tutorial, you will learn about grpc benchmarking tools. We cover key concepts, practical examples, and best practices to help you master this topic.
gRPC benchmarking tools like ghz, grpc-go's benchmark suite, and custom load generators measure latency, throughput, and resource usage of gRPC services under various load patterns and message sizes.
What You'll Learn
- Using ghz for gRPC load testing
- Interpreting latency distributions and throughput
- Benchmarking with different payload sizes
- Comparing protobuf Serialization performance
- Custom benchmark clients for specific scenarios
- Benchmarking streaming RPCs
Why It Matters
Without benchmarks, you can't tell if a change improves or degrades performance. Regular benchmarking catches regressions and validates optimizations. DodaTech's Durga Antivirus Pro benchmarks every gRPC service on every commit, alerting if p99 latency increases by more than 10%.
Real-World Use
A developer changes the protobuf definition for a frequently called service. The benchmark shows p99 latency increased from 50ms to 200ms. The change is reverted before it reaches production, saving an incident.
flowchart LR
A["ghz Load Tester"] --> B["gRPC Service"]
B --> C["Latency Results"]
B --> D["Throughput Results"]
B --> E["Error Rate"]
C --> F["Histogram\np50/p95/p99"]
D --> G["Requests/sec"]
E --> H["Error %"]
F --> I["Compare with\nBaseline"]
I --> J{"Regression > 10%?"}
J -->|Yes| K["Alert: Block PR"]
J -->|No| L["Pass"]
Code Examples
Example 1: ghz Load Testing
# Install ghz
go install github.com/bojand/ghz@latest
# Basic unary benchmark
ghz \
--insecure \
--proto ./proto/threat/v1/threat.proto \
--call threat.v1.ThreatService/ReportThreat \
-d '{"device_id": "dev-001", "threat_name": "Test"}' \
-c 50 \ # 50 concurrent connections
-n 10000 \ # 10,000 total requests
localhost:50051
# With metadata
ghz \
--insecure \
--proto ./threat.proto \
--call threat.v1.ThreatService/ReportThreat \
-d '{"threat_name": "Ransomware"}' \
-m '{"authorization": "Bearer token123"}' \
-c 100 \
-n 50000 \
--timeout 5s \
localhost:50051
# Different payload sizes
ghz \
--insecure \
--proto ./threat.proto \
--call threat.v1.ThreatService/ReportThreat \
-d '{"threat_name": "'"$(python -c 'print("A"*1024)')"'"}' \
-c 20 -n 1000 \
localhost:50051
# Output format
ghz --format=html --output=report.html \
--insecure --proto ./threat.proto \
--call threat.v1.ThreatService/ReportThreat \
-d '{}' \
-c 50 -n 10000 \
localhost:50051
Example 2: Custom Benchmark Client in Go
package main
import (
"context"
"fmt"
"sync"
"time"
"golang.org/x/time/rate"
)
type BenchmarkResult struct {
Latencies []time.Duration
Errors int
Total time.Duration
}
func runBenchmark(client pb.ThreatServiceClient,
numRequests int, concurrency int) *BenchmarkResult {
var wg sync.WaitGroup
results := &BenchmarkResult{}
var mu sync.Mutex
start := time.Now()
sem := make(chan struct{}, concurrency)
for i := 0; i < numRequests; i++ {
wg.Add(1)
sem <- struct{}{}
go func() {
defer wg.Done()
defer func() { <-sem }()
reqStart := time.Now()
_, err := client.ReportThreat(
context.Background(),
&pb.ThreatRequest{
DeviceId: fmt.Sprintf("dev-%d", i%100),
ThreatName: "Benchmark-Threat",
},
)
latency := time.Since(reqStart)
mu.Lock()
results.Latencies = append(results.Latencies,
latency)
if err != nil {
results.Errors++
}
mu.Unlock()
}()
}
wg.Wait()
results.Total = time.Since(start)
return results
}
func analyzeResults(r *BenchmarkResult) {
sort.Slice(r.Latencies, func(i, j int) bool {
return r.Latencies[i] < r.Latencies[j]
})
n := len(r.Latencies)
fmt.Printf("Requests: %d\n", n)
fmt.Printf("Errors: %d (%.2f%%)\n",
r.Errors, float64(r.Errors)/float64(n)*100)
fmt.Printf("Total time: %v\n", r.Total)
fmt.Printf("Throughput: %.0f req/s\n",
float64(n)/r.Total.Seconds())
fmt.Printf("Min: %v\n", r.Latencies[0])
fmt.Printf("P50: %v\n", r.Latencies[n/2])
fmt.Printf("P95: %v\n", r.Latencies[n*95/100])
fmt.Printf("P99: %v\n", r.Latencies[n*99/100])
fmt.Printf("Max: %v\n", r.Latencies[n-1])
}
Example 3: Python Benchmark with Locust
from locust import User, task, between
import grpc
class GrpcBenchUser(User):
wait_time = between(0.1, 0.5)
def __init__(self, environment):
super().__init__(environment)
self.channel = grpc.insecure_channel("localhost:50051")
self.stub = pb.ThreatServiceStub(self.channel)
@task
def report_threat(self):
request = pb.ThreatRequest(
device_id="dev-001",
threat_name="Benchmark-Threat",
)
start = time.time()
try:
response = self.stub.ReportThreat(
request, timeout=5)
latency = (time.time() - start) * 1000
self.environment.events.request.fire(
request_type="grpc",
name="ReportThreat",
response_time=latency,
response_length=len(
response.SerializeToString()),
exception=None,
)
except grpc.RpcError as e:
self.environment.events.request.fire(
request_type="grpc",
name="ReportThreat",
response_time=0,
response_length=0,
exception=e,
)
def stop(self):
self.channel.close()
super().stop()
Common Mistakes
- Benchmarking on the same machine — network latency and CPU contention differ between local and production. Benchmark in a realistic network environment.
- Ignoring warmup — the first 100-1000 requests establish connections and warm caches. Discard them from results.
- Testing one payload size only — performance varies dramatically with message size. Test at 100B, 1KB, 10KB, 100KB, and 1MB.
- Not measuring tail latency — average latency hides problems. Always report p50, p95, p99, and max latency.
- Benchmarking without Load Balancing — test with multiple server instances to measure real-world load balancing behavior.
Practice Questions
- What metrics should a gRPC benchmark report?
- How does concurrency level affect throughput and latency?
- Why should you discard warmup requests from results?
- What is the difference between throughput and goodput?
- How do you benchmark streaming RPCs?
Challenge: Design a benchmark suite that tests a gRPC service under 5 different load patterns: Steady State, burst, ramp-up, constant high load, and recovery after overload. Measure how the service behaves under each pattern.
Mini Project
Build a gRPC benchmark automation tool that: runs ghz with configurable parameters (concurrency, payload size, duration), stores results in a time-series database, compares against baselines, generates HTML reports with latency histograms, and alerts on regressions.
FAQ
What's Next
Learn more about gRPC performance optimization
Built by the developers of DodaTech
Doda Browser, DodaZIP & Durga Antivirus Pro