gRPC Load Balancing — Complete Guide

In this tutorial, you will learn about grpc load balancing. We cover key concepts, practical examples, and best practices to help you master this topic.

gRPC load balancing distributes RPC calls across multiple server instances. Unlike HTTP/1.1, gRPC's long-lived HTTP/2 connections require careful load balancing strategies to avoid connection concentration.

What You'll Learn

• Client-side vs server-side load balancing
• Built-in load balancing policies (round_robin, pick_first)
• Service discovery integration
• Subchannel management
• Load balancing for streaming RPCs

Why It Matters

Without proper load balancing, all gRPC traffic may route to a single server instance. Client-side load balancing is preferred for gRPC because HTTP/2 multiplexing makes per-request proxying inefficient.

Real-World Use

Kubernetes uses headless services with client-side round-robin for gRPC. gRPC's resolver and load balancer APIs integrate with etcd, Consul, and Envoy. Google Cloud uses gRPC load balancing for all internal services.

flowchart LR
    Client[gRPC Client] --> Resolver[Name Resolver]
    Resolver --> Addresses[Server Addresses]
    Addresses --> LB[Load Balancer Policy]
    LB --> Server1[Server Instance 1]
    LB --> Server2[Server Instance 2]
    LB --> Server3[Server Instance 3]
    Server1 --> Subchannel1[Subchannel]
    Server2 --> Subchannel2[Subchannel]
    Server3 --> Subchannel3[Subchannel]

Teacher Mindset

Client-side load balancing gives the client direct control over which server receives each RPC. Each RPC can go to any server. For streaming RPCs, all messages in a stream go to the same server.

Code Examples

// Example 1: Client-side round-robin load balancing
const grpc = require('@grpc/grpc-js');

const client = new OrderService(
  'dns:///orderservice.example.com:50051',
  grpc.credentials.createSsl(caCert),
  {
    'grpc.lb_policy_name': 'round_robin',
    'grpc.service_config': JSON.stringify({
      loadBalancingConfig: [{ round_robin: {} }]
    })
  }
);

// Example 2: Custom resolver for service discovery
const customResolver = {
  class: class CustomResolver {
    constructor(target, listener) {
      this.listener = listener;
    }

    async updateResolution() {
      const addresses = await serviceDiscovery.getEndpoints('order-service');
      this.listener.onSuccessfulResolution(
        addresses.map(addr => ({
          address: addr.host,
          port: addr.port
        })),
        null,
        null,
        { serviceConfig: null }
      );
    }

    destroy() {}
  }
};

grpc.resolverRegistry.register('custom', customResolver);

# Example 3: Kubernetes headless service for gRPC
apiVersion: v1
kind: Service
metadata:
  name: order-service-grpc
spec:
  clusterIP: None  # Headless service
  selector:
    app: order-service
  ports:
    - port: 50051
      name: grpc
---
apiVersion: v1
kind: Service
metadata:
  name: order-service
spec:
  selector:
    app: order-service
  ports:
    - port: 50051
      name: grpc
  type: ClusterIP

Common Mistakes

• Using traditional load balancers (NLB/ALB) that terminate HTTP/2 and break streaming
• Not configuring health checks that are gRPC-aware (use gRPC health probe)
• Using pick_first policy (default) without understanding that all RPCs go to one server
• Forgetting that client-side load balancing requires service discovery
• Not handling subchannel state changes and reconnection

Practice

1. Start two gRPC server instances on different ports.
2. Configure a client with round_robin load balancing.
3. Verify that RPCs are distributed across both servers.
4. Stop one server and observe client failover.
5. Challenge: Implement a weighted round_robin policy where servers have different capacity.

FAQ

Mini Project

Deploy your order service with two instances. Implement client-side round_robin load balancing. Add health checking. Test failover by stopping one instance and verifying RPCs route to the remaining instance.

What's Next

Next, you will learn about gRPC Reflection for server introspection without needing the proto file.