Load Balancing Web Servers: Complete Guide

Load balancing distributes incoming network traffic across multiple backend servers to improve availability, scalability, and fault tolerance. A load balancer acts as a single entry point, routing requests to healthy servers while isolating failed ones.

In this tutorial, you will learn load balancing algorithms (round-robin, least connections, IP hash), configure health checks, implement session persistence, set up SSL termination at the load balancer, and design horizontal scaling strategies. DodaTech uses load balancing to distribute traffic across Doda Browser update servers, DodaZIP download mirrors, and Durga Antivirus Pro signature distribution infrastructure.

What You'll Learn

By the end of this guide, you will understand load balancing algorithms and when to use each, configure active and passive health checks, implement session persistence for stateful applications, and design a horizontally scalable web server architecture.

Why Load Balancing Matters

A single server is a single point of failure. Load balancing eliminates this by distributing traffic across multiple servers. As traffic grows, you add more servers behind the load balancer without changing the public endpoint. This horizontal scaling approach is the foundation of high-availability web architecture. Every Web Servers administrator and DevOps engineer must understand load balancing concepts to build reliable systems.

Load Balancing Learning Path

flowchart LR
  A[Algorithms] --> B[Health Checks]
  B --> C[Session Persistence]
  C --> D[SSL Termination]
  D --> E[Horizontal Scaling]
  E --> F{You Are Here}
  style F fill:#f90,color:#fff

Load Balancing Algorithms

The choice of algorithm depends on your application's characteristics:

Round-robin with NGINX

upstream backend {
    # Default: round-robin
    server 10.0.1.1:80;
    server 10.0.1.2:80;
    server 10.0.1.3:80;
}

server {
    listen 80;
    server_name app.dodatech.com;

    location / {
        proxy_pass http://backend;
        proxy_set_header Host $host;
        proxy_set_header X-Real-IP $remote_addr;
    }
}

Least connections with HAProxy

backend web_servers
    balance leastconn
    option httpchk GET /health
    default-server inter 3000 fall 3 rise 2

    server web1 10.0.1.1:80 check weight 10
    server web2 10.0.1.2:80 check weight 10
    server web3 10.0.1.3:80 check weight 5

Testing distribution

# Send 10 requests and observe distribution
for i in $(seq 1 10); do
  curl -s http://app.dodatech.com/ | grep "Server ID"
done

# Expected: requests distributed across backend servers
# Server ID: web1
# Server ID: web2
# Server ID: web3
# ...

Health Checks

Active health checks periodically test backend servers:

upstream backend {
    server 10.0.1.1:80 max_fails=3 fail_timeout=30s;
    server 10.0.1.2:80 max_fails=3 fail_timeout=30s;
    server 10.0.1.3:80 backup;

    # Keepalive connections to backends
    keepalive 32;
}

server {
    listen 80;
    server_name app.dodatech.com;

    location / {
        proxy_pass http://backend;
        proxy_http_version 1.1;
        proxy_set_header Connection "";
    }
}

With HAProxy (more sophisticated health checks):

backend api_servers
    balance roundrobin

    # HTTP health check with expected status
    option httpchk GET /health HTTP/1.1\r\nHost:\ api.dodatech.com
    http-check expect status 200
    http-check expect string "healthy"

    # Mark server down after 3 failures within 2 seconds
    default-server inter 2000 fall 3 rise 2

    server api1 10.0.2.1:3000 check
    server api2 10.0.2.2:3000 check
    server api3 10.0.2.3:3000 check

    # Optionally: mark servers based on layer 7 checks
    option allbackups

Expected health check behavior

# When a server fails health checks:
# HAProxy marks it DOWN and stops routing traffic
# NGINX marks it unavailable after max_fails failures

# Check server status
echo "show stat" | sudo socat stdio /var/run/haproxy/admin.sock
# api_servers api1 UP 1 1 0 0 0
# api_servers api2 DOWN 0 1 3 120  (failed 3 times, 120s downtime)

Session Persistence

Stateful applications need requests from the same client to reach the same server:

Cookie-based (NGINX with sticky module)

upstream backend {
    server 10.0.1.1:80;
    server 10.0.1.2:80;

    # Sticky cookie (requires NGINX Plus or open-source sticky module)
    sticky cookie srv_id expires=1h domain=.dodatech.com path=/;
}

Source IP hash

upstream backend {
    ip_hash;
    server 10.0.1.1:80;
    server 10.0.1.2:80;
    server 10.0.1.3:80;
}

Cookie-based with HAProxy

backend app_servers
    balance roundrobin
    # Insert a cookie that ties the client to a specific server
    cookie SERVERID insert indirect nocache
    server app1 10.0.4.1:8080 cookie app1 check
    server app2 10.0.4.2:8080 cookie app2 check
    server app3 10.0.4.3:8080 cookie app3 check

Testing persistence

# First request gets a cookie
curl -c cookies.txt -b cookies.txt http://app.dodatech.com/login
# Response includes: Set-Cookie: SERVERID=app1

# Subsequent requests with the cookie go to the same server
curl -c cookies.txt -b cookies.txt http://app.dodatech.com/dashboard
# Routes to app1 (same server as login)

SSL Termination at the Load Balancer

Terminating SSL at the load balancer reduces CPU load on backend servers:

server {
    listen 443 ssl http2;
    server_name app.dodatech.com;

    ssl_certificate /etc/letsencrypt/live/app.dodatech.com/fullchain.pem;
    ssl_certificate_key /etc/letsencrypt/live/app.dodatech.com/privkey.pem;
    ssl_protocols TLSv1.2 TLSv1.3;
    ssl_ciphers ECDHE-ECDSA-AES128-GCM-SHA256:ECDHE-RSA-AES128-GCM-SHA256;

    # Forward client IP and protocol to backend
    proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
    proxy_set_header X-Forwarded-Proto https;
    proxy_set_header X-Real-IP $remote_addr;

    location / {
        proxy_pass http://backend;
    }
}

server {
    listen 80;
    server_name app.dodatech.com;
    return 301 https://$server_name$request_uri;
}

With HAProxy

frontend https_frontend
    bind *:443 ssl crt /etc/haproxy/certs/dodatech.pem

    # Forwarded headers
    option forwardfor
    http-request set-header X-Forwarded-Proto https if { ssl_fc }

    default_backend web_servers

backend web_servers
    balance roundrobin
    server web1 10.0.1.1:80 check
    server web2 10.0.1.2:80 check

Horizontal Scaling Strategy

A horizontally scaled architecture:

flowchart LR
    LB1[Load Balancer] --> A[Web Server 1]
    LB1 --> B[Web Server 2]
    LB1 --> C[Web Server 3]
    A --> DB[(Database)]
    B --> DB
    C --> DB
    A --> S3[(Shared Storage)]
    B --> S3
    C --> S3

Key considerations for horizontal scaling:

# 1. All servers must share the same application code
# Use a deployment tool like Ansible or Terraform

# 2. Sessions must be stored in a shared backend (Redis/Memcached)
# Not in local memory

# 3. Uploaded files must go to shared storage (S3/NFS)
# Not stored locally

# 4. Health checks must verify full application stack
# Not just process liveness

# Example: Add a new server to the pool
# 1. Provision server with configuration management
ansible-playbook -i inventory/production deploy-web.yml --limit new-server

# 2. Add to load balancer
echo "server new-server 10.0.1.4:80 check" >> /etc/haproxy/haproxy.cfg
sudo systemctl reload haproxy

# 3. Verify traffic reaches the new server
curl -I http://app.dodatech.com

Common Errors

1. All Backend Servers Marked Down

Health checks are failing. Check the health endpoint on each server. Ensure the health check URL and expected status match the application's response.

2. Session Lost Between Requests

Session persistence is not configured or the session store is not shared. Use cookie-based persistence at the load balancer and a shared Redis session store.

3. Uneven Traffic Distribution

The load balancing algorithm does not match the workload. For long-lived connections, use least connections instead of round-robin. Check server capacity weighting.

4. SSL Termination Overload

The load balancer is struggling with SSL termination. Offload SSL to dedicated hardware, reduce cipher complexity, or add more load balancer instances.

5. Backend Server Showing Stale Data

The server was removed from the pool but still receives requests due to DNS caching or persistent connections. Gracefully drain connections before removing a server.

Practice Questions

1. What is the difference between round-robin and least connections load balancing? Round-robin distributes requests sequentially regardless of server load. Least connections sends requests to the server with the fewest active connections, which is better when request processing time varies.

2. Why is session persistence needed in a load-balanced environment? Without persistence, a user's subsequent requests may go to different servers. If session data is stored locally, the user's state is lost. Persistence ensures all requests from a client reach the same server.

3. How does a health check prevent downtime? Health checks continuously verify that backend servers are responding correctly. When a server fails health checks, the load balancer automatically removes it from the pool, preventing requests from being routed to a non-functional server.

4. Challenge: Multi-tier load balancing design

Design a load balancing architecture for an e-commerce application:

• Two web servers serving static content and PHP
• Two API servers running Node.js
• One admin panel server (internal access only)
• SSL termination at the load balancer
• Session persistence for the cart
• Separate health checks for each tier

Mini Project: High-Availability Load Balancing Stack

Build a production-grade load balancing infrastructure:

1. Set up three NGINX web servers serving a static application
2. Configure HAProxy as the load balancer with:
   • Round-robin distribution
   • Active HTTP health checks every 3 seconds
   • Cookie-based session persistence
   • SSL termination with Let's Encrypt
   • Stats dashboard on port 8404
3. Test failover by stopping one web server
4. Add a fourth web server to the pool without downtime
5. Monitor traffic distribution through the stats dashboard

# Simulate failover test
sudo systemctl stop nginx  # On web1

# Verify traffic still flows
curl -I https://app.dodatech.com

# Check load balancer stats
curl -u admin:password http://localhost:8404/stats | grep "web_servers"

# Restart the failed server
sudo systemctl start nginx  # On web1

# Web1 should automatically rejoin the pool

This architecture mirrors how DodaTech provides high-availability infrastructure for Doda Browser update distribution and Durga Antivirus Pro signature delivery across multiple geographic regions.

FAQ

Built by the developers of Doda Browser, DodaZIP, and Durga Antivirus Pro.