Cache Basics: Layers, Policies, and Key Concepts

In this tutorial, you will learn about Cache Basics: Layers, Policies, and Key Concepts. We cover key concepts, practical examples, and best practices to help you master this topic.

Caching operates at multiple layers in a system: client-side (browser), network (CDN, reverse proxy), application (in-memory), and database (query cache, buffer pool). Each layer has distinct characteristics in speed, capacity, and eviction strategy, and understanding these layers helps you decide where to cache specific data.

flowchart TB
    subgraph Client
        BrowserCache
    end
    subgraph Network
        CDN[CDN / Reverse Proxy]
    end
    subgraph Application
        AppCache[In-Memory Cache]
    end
    subgraph Database
        BufferPool[Buffer Pool / Query Cache]
    end
    BrowserCache --> CDN
    CDN --> AppCache
    AppCache --> BufferPool

What You'll Learn

• The four primary caching layers and their trade-offs
• Eviction policies: LRU, LFU, FIFO, TTL-based, and random
• Cache admission policies and working set estimation

Why It Matters

Choosing the wrong cache layer or eviction policy leads to poor hit rates, wasted memory, and unpredictable performance. Understanding the fundamentals ensures you design a cache that behaves predictably under varying workloads.

Real-World Use

A video streaming service uses browser caching for static assets (JS, CSS), CDN caching for video segments, application caching for user recommendations, and database caching for metadata queries. Each layer has a different TTL and eviction policy tuned to the data's access pattern.

Eviction Policies

LRU (Least Recently Used)

class LRUCache {
  constructor(capacity) {
    this.capacity = capacity;
    this.cache = new Map();
  }

  get(key) {
    if (!this.cache.has(key)) return -1;
    const value = this.cache.get(key);
    this.cache.delete(key);
    this.cache.set(key, value);
    return value;
  }

  put(key, value) {
    if (this.cache.has(key)) this.cache.delete(key);
    else if (this.cache.size >= this.capacity) {
      const oldest = this.cache.keys().next().value;
      this.cache.delete(oldest);
    }
    this.cache.set(key, value);
  }
}

Expected output:

LRUCache with capacity 3: after putting A, B, C, then D, key A is evicted (least recently used).

LFU (Least Frequently Used)

class LFUCache {
  constructor(capacity) {
    this.capacity = capacity;
    this.cache = new Map();
    this.freq = new Map();
  }

  get(key) {
    if (!this.cache.has(key)) return -1;
    this.freq.set(key, (this.freq.get(key) || 0) + 1);
    return this.cache.get(key);
  }

  put(key, value) {
    if (this.capacity === 0) return;
    if (this.cache.has(key)) {
      this.cache.set(key, value);
      this.freq.set(key, (this.freq.get(key) || 0) + 1);
      return;
    }
    if (this.cache.size >= this.capacity) {
      let minFreq = Infinity;
      let evictKey = null;
      for (const [k, f] of this.freq) {
        if (f < minFreq) { minFreq = f; evictKey = k; }
      }
      this.cache.delete(evictKey);
      this.freq.delete(evictKey);
    }
    this.cache.set(key, value);
    this.freq.set(key, 1);
  }
}

Expected output:

LFUCache evicts the least frequently accessed item. A frequently accessed key survives even if it was added early.

TTL-Based Eviction

function createTTLCache(defaultTTL = 60000) {
  const store = new Map();

  return {
    get(key) {
      const entry = store.get(key);
      if (!entry) return null;
      if (Date.now() > entry.expires) {
        store.delete(key);
        return null;
      }
      return entry.value;
    },
    set(key, value, ttl = defaultTTL) {
      store.set(key, { value, expires: Date.now() + ttl });
    },
    delete(key) { store.delete(key); },
    size() { return store.size; }
  };
}

Expected output:

After TTL expires, get returns null and the entry is lazily evicted on next access.

Common Mistakes

• Using LRU for workloads with frequent bulk scans that pollute the cache with one-time-use data.
• Setting the same TTL for all cache entries regardless of data volatility.
• Over-provisioning cache memory without monitoring actual hit rate — bigger is not always better.
• Ignoring the cost of serialization: caching large objects with JSON.stringify adds latency on every write and read.
• Not considering cache Sharding or Partitioning in Distributed Systems, causing hot keys on a single node.

Practice Questions

1. Which eviction policy suits a workload where recently accessed items are likely to be accessed again soon?
2. What is the difference between cache eviction and cache invalidation?
3. How does a CDN caching layer differ from an application-level cache?
4. Why might you choose FIFO over LRU for a specific use case?
5. What is the working set of a cache?

Challenge

You have a 10GB cache for a photo-sharing app. Users upload new photos every second, and popular photos from last week are still viewed frequently. Design a policy that balances recency and frequency without starving new content.

FAQ

Mini Project

Extend the intro project: implement an LRU cache with configurable capacity for your blog API. Add cache stats middleware that logs hit rate, miss rate, and eviction count. Compare behavior under a burst of 10,000 requests to 1,000 unique post IDs.

What's Next

Now explore Client-Side Caching to learn how browsers and mobile apps cache responses for offline support and faster page loads.