gRPC Compression — Reducing Payload Size with Gzip, Snappy, and Custom Codecs

In this tutorial, you will learn about grpc compression. We cover key concepts, practical examples, and best practices to help you master this topic.

gRPC compression reduces payload size by compressing protobuf messages before sending them over the network, using codecs like gzip, Snappy, or Zstandard to balance compression ratio against CPU cost.

What You'll Learn

• Enabling gzip compression in gRPC clients and servers
• Configuring compression levels
• Using Snappy and Zstandard codecs
• Per-call compression settings
• Measuring bandwidth savings and CPU overhead

Why It Matters

Protobuf already reduces payload size compared to JSON, but for large messages — like threat reports with logs or device inventories — compression can reduce bandwidth by 5-10x. DodaTech's Durga Antivirus Pro uses Snappy compression on threat report uploads, reducing 50MB reports to 4MB and cutting bandwidth costs by 90%.

Real-World Use

A security agent on an IoT device sends hourly threat logs to the server. Each message is 2MB of JSON-like protobuf data. With Snappy compression, the payload drops to 200KB, reducing the device's data usage from 48MB/day to under 5MB/day — critical for devices on metered connections.

flowchart LR
    A["Client\nProtobuf Message"] --> B["Compression\nCodec"]
    B --> C{"Which Codec?"}
    C -->|gzip| D["gzip: 90% reduction\nHigh CPU"]
    C -->|Snappy| E["Snappy: 80% reduction\nLow CPU"]
    C -->|Zstandard| F["Zstd: 85% reduction\nMedium CPU"]
    D --> G["Network"]
    E --> G
    F --> G
    G --> H["Decompression\nCodec"]
    H --> I["Server\nProtobuf Message"]
    style B fill:#fef3c7,stroke:#d97706
    style H fill:#fef3c7,stroke:#d97706

Code Examples

Example 1: Enabling gzip Compression in Go

package main

import (
    "google.golang.org/grpc"
    "google.golang.org/grpc/encoding/gzip"
)

func startServer() {
    // Server accepts compressed messages
    s := grpc.NewServer(
        grpc.RPCCompressor(grpc.NewGZIPCompressor()),
        grpc.RPCDecompressor(grpc.NewGZIPDecompressor()),
    )
    pb.RegisterThreatServiceServer(s, &server{})
}

func startClient() {
    // Client sends compressed messages
    conn, _ := grpc.Dial("localhost:50051",
        grpc.WithInsecure(),
        grpc.WithDefaultCallOptions(
            grpc.UseCompressor(gzip.Name),
        ),
    )
    client := pb.NewThreatServiceClient(conn)
}

// Per-call compression
response, err := client.ReportThreat(
    ctx,
    request,
    grpc.UseCompressor(gzip.Name),
)

Example 2: Snappy Compression in Python

import grpc
import snappy

class SnappyCompressor(grpc.Compression):
    @staticmethod
    def compress(data):
        return snappy.compress(data)
    
    @staticmethod
    def decompress(data):
        return snappy.decompress(data)
    
    name = "snappy"

# Server with Snappy
server = grpc.server(
    grpc.insecure_server(),
    compression=SnappyCompressor,
)

# Client with Snappy
channel = grpc.insecure_channel(
    "localhost:50051",
    options=[
        ("grpc.default_compression_algorithm", 3),  # 3=Snappy
    ],
)

# Measure compression ratio
original_size = len(request.SerializeToString())
compressed = SnappyCompressor.compress(
    request.SerializeToString())
compressed_size = len(compressed)
ratio = compressed_size / original_size * 100
print(f"Original: {original_size} bytes")
print(f"Compressed: {compressed_size} bytes")
print(f"Ratio: {ratio:.1f}%")

Example 3: Compression Level Configuration

const grpc = require('@grpc/grpc-js');
const zlib = require('zlib');

// Custom compression with level control
class LevelCompressor {
  constructor(level = 6) {
    this.level = level;
  }
  
  compress(data) {
    return zlib.gzipSync(data, { level: this.level });
  }
  
  decompress(data) {
    return zlib.gunzipSync(data);
  }
  
  get name() {
    return 'gzip';
  }
}

// Fast compression (level 1) for latency-sensitive calls
const fastCompressor = new LevelCompressor(1);

// Maximum compression (level 9) for batch/offline calls
const maxCompressor = new LevelCompressor(9);

// Different compression per message type
function getCompressor(messageType, size) {
  if (size > 1024 * 1024) {
    return maxCompressor; // Large messages: max compression
  }
  if (messageType === 'RealtimeUpdate') {
    return fastCompressor; // Real-time: fast compression
  }
  return null; // Small messages: no compression
}

Common Mistakes

1. Compressing already-compressed data — protobuf binary data doesn't compress as well as text. Test compression ratios before enabling globally.
2. Using gzip on very small messages — messages under 1KB can actually get larger after gzip due to header overhead. Use Snappy or skip compression for small payloads.
3. Not measuring CPU impact — gzip level 9 can use 10x more CPU than Snappy. Measure both sides (client and server) to find the right balance.
4. Forgetting to configure decompression — if the server enables compression but doesn't register the decompressor, it will fail to read compressed messages.
5. Assuming symmetric compression — a slow client might benefit from fast compression (Snappy) while the server decompresses. Choose based on your bottleneck.

Practice Questions

1. How does gzip compression compare to Snappy in terms of ratio and speed?
2. Why might compression not help for very small messages?
3. How do you configure different compression algorithms per call?
4. What is the trade-off between compression level and CPU usage?
5. How do you measure the real-world bandwidth savings of compression?

Challenge: Build a compression benchmarking tool that tests gzip (levels 1-9), Snappy, and Zstandard on real threat report data, measuring compression ratio, client CPU time, server CPU time, and end-to-end latency for each.

Mini Project

Implement an adaptive compression system for a gRPC service that: uses no compression for messages under 1KB, Snappy for messages under 100KB, and gzip level 6 for larger messages. Log compression ratios and automatically disable compression if CPU exceeds 80%.

FAQ

What's Next

Learn about gRPC streaming patterns