In-Memory Cache: Local Caching with Runtime Data Stores

In this tutorial, you will learn about In. We cover key concepts, practical examples, and best practices to help you master this topic.

In-memory caching stores data directly in the application Process's heap. It is the fastest cache layer — no network round-trip, no Serialization overhead for native objects. However, it is per-instance (not shared) and limited by available RAM.

flowchart LR
    subgraph Instance1[App Instance 1]
        L1_1[L1 In-Memory Cache]
        App1[Application]
    end
    subgraph Instance2[App Instance 2]
        L1_2[L1 In-Memory Cache]
        App2[Application]
    end
    Instance1 --> Redis[Distributed Cache - L2]
    Instance2 --> Redis
    Redis --> DB[Database]

What You'll Learn

• In-memory cache patterns using Maps, WeakMaps, and LRU libraries
• TTL-based and size-based eviction for local caches
• Two-tier caching: local L1 cache backed by distributed L2 cache
• Thread/process safety considerations for in-memory caches

Why It Matters

In-memory caches have the lowest access latency (nanoseconds to microseconds). A two-tier cache (local + Redis) reduces Redis load by 60-80% because hot keys are served from the local instance without any network call.

Real-World Use

A microservice handling product search uses a local LRU cache for popular search terms. The cache holds 10,000 entries with 60-second TTL. It reduces calls to the shared Redis cluster by 75%, allowing Redis to scale horizontally for less popular queries.

In-Memory Cache Implementations

TTL-Based Cache with Map

class TTLMap {
  constructor(defaultTTL = 30000) {
    this.store = new Map();
    this.defaultTTL = defaultTTL;
  }

  get(key) {
    const entry = this.store.get(key);
    if (!entry) return null;
    if (Date.now() > entry.expires) {
      this.store.delete(key);
      return null;
    }
    return entry.value;
  }

  set(key, value, ttl) {
    const expires = Date.now() + (ttl || this.defaultTTL);
    this.store.set(key, { value, expires });
  }

  delete(key) { this.store.delete(key); }

  clear() { this.store.clear(); }

  get size() { return this.store.size; }
}

Expected output:

Keys are automatically evicted lazily on read after TTL expires. Memory is reclaimed only when keys are accessed or deleted.

LRU Cache Using Library (lru-cache)

const LRU = require('lru-cache');

const cache = new LRU.LRUCache({
  max: 500,
  maxAge: 1000 * 60 * 5,
  dispose(key, n) { console.log(`Evicted ${key}`); }
});

function getExpensiveData(key) {
  if (cache.has(key)) return cache.get(key);
  const data = computeExpensiveThing(key);
  cache.set(key, data);
  return data;
}

Expected output:

Cache holds max 500 entries. Entries older than 5 minutes are evicted. Eviction events are logged.

Two-Tier (L1 + L2) Cache

class TwoTierCache {
  constructor(l1Size = 100, l2TTL = 3600) {
    this.l1 = new LRU.LRUCache({ max: l1Size, maxAge: 30000 });
    this.l2 = redisClient;
    this.l2TTL = l2TTL;
  }

  async get(key, fetchFn) {
    const l1Hit = this.l1.get(key);
    if (l1Hit !== undefined) return l1Hit;

    const l2Value = await this.l2.get(key);
    if (l2Value) {
      const parsed = JSON.parse(l2Value);
      this.l1.set(key, parsed);
      return parsed;
    }

    const data = await fetchFn();
    this.l1.set(key, data);
    await this.l2.setEx(key, this.l2TTL, JSON.stringify(data));
    return data;
  }

  async invalidate(key) {
    this.l1.delete(key);
    await this.l2.del(key);
  }
}

Expected output:

L1 hit: returns in <1μs. L2 hit: returns in ~1ms, populates L1. Miss: fetches from origin, populates both caches.

Common Mistakes

• Using in-memory cache for data that must be consistent across instances, causing each instance to show different values.
• Not setting a size or TTL limit on in-memory caches, causing OutOfMemoryError under load.
• Storing mutable objects in cache — if the caller modifies the returned object, the cache is corrupted.
• Ignoring Garbage Collection pressure — in-memory caches with millions of entries cause long GC pauses.
• Making the L1 cache too large, reducing memory available for the application's core logic.

Practice Questions

1. What is the main advantage of an in-memory cache over a distributed cache like Redis?
2. When is an in-memory cache inappropriate?
3. How does the two-tier cache pattern improve performance and consistency?
4. Why should you avoid storing mutable objects in a cache?
5. What is the relationship between in-memory cache size and GC pause time?

Challenge

Design an in-memory cache for a stock ticker service that receives 1000 price updates per second. The cache must serve sub-microsecond reads, handle writes from a single updater thread, and expire stale entries after 5 seconds. Ensure the cache does not grow unbounded.

FAQ

Mini Project

Extend the blog API with a two-tier cache: L1 is an in-memory LRU cache (500 entries, 30s TTL), L2 is Redis (3600s TTL). Add metrics middleware tracking L1 hit, L2 hit, and miss rates. Run a load test comparing single-tier Redis caching against two-tier.

What's Next

Continue with Database Caching to learn about query caching, materialized views, and database-level caching strategies.