Distributed Cache: Sharding, Replication, and Consistency at Scale

In this tutorial, you will learn about Distributed Cache: Sharding, Replication, and Consistency at Scale. We cover key concepts, practical examples, and best practices to help you master this topic.

A distributed cache spans multiple nodes to provide larger capacity, higher throughput, and fault tolerance. Data is partitioned across nodes using sharding strategies, and replication ensures availability when nodes fail. Distributed caches introduce challenges around consistency, coordination, and network partitions.

flowchart TB
    Client1[Client] --> Proxy[Cache Proxy / Router]
    Client2[Client] --> Proxy
    Proxy -->|hash(key) % N| Node1[Cache Node 1]
    Proxy -->|hash(key) % N| Node2[Cache Node 2]
    Proxy -->|hash(key) % N| Node3[Cache Node 3]
    Node1 ---|Replication| Node2
    Node2 ---|Replication| Node3
    Node3 ---|Replication| Node1

What You'll Learn

• Consistent hashing for cache node sharding
• Replication strategies: master-replica, multi-master
• Consistency Models: eventual, strong, read-your-writes
• Failure detection and cache rebalancing

Why It Matters

A single cache node has finite memory and throughput. Distributed caches scale horizontally to terabytes of cached data and millions of operations per second. However, poor sharding design leads to hot spots and uneven load distribution.

Real-World Use

A global e-commerce platform uses Redis Cluster with 12 shards across 6 nodes. Products are sharded by category ID using consistent hashing. Each shard has a replica in a different availability zone. A node failure causes no data loss and minimal rebalancing.

Distributed Cache Implementations

Consistent Hashing for Cache Routing

class ConsistentHash {
  constructor(nodes = [], replicas = 3) {
    this.replicas = replicas;
    this.ring = new Map();
    this.sortedKeys = [];

    for (const node of nodes) {
      this.addNode(node);
    }
  }

  addNode(node) {
    for (let i = 0; i < this.replicas; i++) {
      const hash = this.hash(`${node}:${i}`);
      this.ring.set(hash, node);
      this.sortedKeys.push(hash);
    }
    this.sortedKeys.sort((a, b) => a - b);
  }

  removeNode(node) {
    for (let i = 0; i < this.replicas; i++) {
      const hash = this.hash(`${node}:${i}`);
      this.ring.delete(hash);
    }
    this.sortedKeys = Array.from(this.ring.keys()).sort((a, b) => a - b);
  }

  getNode(key) {
    if (this.sortedKeys.length === 0) return null;
    const hash = this.hash(key);
    let pos = this.sortedKeys.findIndex(k => k >= hash);
    if (pos === -1) pos = 0;
    return this.ring.get(this.sortedKeys[pos]);
  }

  hash(key) {
    let hash = 5381;
    for (let i = 0; i < key.length; i++) {
      hash = ((hash << 5) + hash) + key.charCodeAt(i);
    }
    return hash >>> 0;
  }
}

Expected output:

Adding/removing a node causes only K/N keys to relocate (where K = total keys, N = number of replicas). Without consistent hashing, removing a node would relocate all keys.

Redis Cluster Client

const Redis = require('ioredis');

const cluster = new Redis.Cluster([
  { host: 'cache-1', port: 6379 },
  { host: 'cache-2', port: 6379 },
  { host: 'cache-3', port: 6379 },
  { host: 'cache-4', port: 6379 }
], {
  scaleReads: 'slave',
  enableReadyCheck: true,
  clusterRetryStrategy: (times) => Math.min(100 + times * 200, 3000)
});

async function getDistributed(key) {
  try {
    return await cluster.get(key);
  } catch (err) {
    if (err.code === 'MOVED' || err.code === 'ASK') {
      return cluster.get(key); // automatically redirects
    }
    throw err;
  }
}

Expected output:

ioredis handles cluster slot routing, MOVED/ASK redirection, and read scaling to replicas. Keys are distributed across 16384 hash slots.

Cache Rebalancing on Node Add/Remove

class CacheRebalancer {
  constructor(ring, sourceRedis, targetRedis) {
    this.ring = ring;
    this.source = sourceRedis;
    this.target = targetRedis;
  }

  async rebalance() {
    const keys = await this.source.keys('*');
    const migrated = [];
    const errors = [];

    for (const key of keys) {
      const targetNode = this.ring.getNode(key);
      if (targetNode !== this.source.targetNode) {
        try {
          const value = await this.source.dump(key);
          await this.target.restore(key, 0, value);
          await this.source.del(key);
          migrated.push(key);
        } catch (err) {
          errors.push({ key, error: err.message });
        }
      }
    }

    return { migrated: migrated.length, errors: errors.length };
  }
}

Expected output:

When a node is added, only keys that should be on the new node are migrated. Keys already on the correct node are untouched.

Common Mistakes

• Using simple modulo hashing (hash(key) % N) instead of consistent hashing — adding or removing a node invalidates the entire cache.
• Not configuring read replicas for read-heavy workloads, overloading the primary shard.
• Ignoring network partitions — a split-brain scenario can cause cache inconsistency.
• Using a single proxy/router that becomes a bottleneck and single point of failure.
• Not monitoring per-node memory and hit rate — a hot shard may be overloaded while others are idle.

Practice Questions

1. How does consistent hashing differ from simple modulo sharding?
2. What happens to cached data when a node fails in a distributed cache?
3. What is the difference between cache sharding and cache replication?
4. How does read-your-writes consistency work in a distributed cache?
5. What are hot spots in a distributed cache and how do you mitigate them?

Challenge

Design a distributed cache for a global social media platform. Use consistent hashing with 32 virtual nodes per physical node. Implement read-from-replica for popular content. Design a rebalancing Strategy that minimizes data movement when scaling from 4 to 8 nodes.

FAQ

Mini Project

Build a distributed cache simulator in Node.js. Implement consistent hashing with configurable virtual nodes. Simulate adding/removing nodes and measure the percentage of keys that need rehashing. Compare against simple modulo hashing. Visualize the distribution of keys across nodes.

What's Next

Continue with Cache Consistency to explore data consistency guarantees between cache and database.