Server-Side Caching: Application-Level and Reverse Proxy Strategies

In this tutorial, you will learn about Server. We cover key concepts, practical examples, and best practices to help you master this topic.

Server-side caching operates on the origin server or in front of it, storing responses in memory (RAM) or on fast local storage so repeated requests for the same resource are served without re-executing application logic or hitting the database.

flowchart TB
    Client --> LB[Load Balancer]
    LB --> RP[Reverse Proxy / CDN]
    RP --> App1[App Instance 1]
    RP --> App2[App Instance 2]
    App1 --> MC[Memory Cache - Redis/Memcached]
    App2 --> MC
    App1 --> DB
    App2 --> DB

What You'll Learn

• In-memory caching with Redis, Memcached, and built-in language runtimes
• Reverse proxy caching with Nginx and Varnish
• Response caching middleware for Express, Fastify, and Koa
• Cache stampede prevention with request coalescing

Why It Matters

Server-side caching reduces response times from 50-200ms to 1-5ms for cached endpoints. It also dramatically reduces database query load — a 90% cache hit rate translates to 10x fewer database connections, delaying or eliminating the need for read replicas.

Real-World Use

A high-traffic API Gateway uses Nginx to cache GET responses for 60 seconds. When a cache key matches, Nginx returns the cached response without forwarding the request to the application. This absorbs traffic spikes during viral events without scaling the application tier.

Cache Stampede Prevention with Request Coalescing

const pendingRequests = new Map();

async function getWithCoalescing(key, fetchFn, ttlMs) {
  if (pendingRequests.has(key)) {
    return pendingRequests.get(key);
  }
  const promise = fetchFn().then((value) => {
    setCache(key, value, ttlMs);
    pendingRequests.delete(key);
    return value;
  }).catch((err) => {
    pendingRequests.delete(key);
    throw err;
  });
  pendingRequests.set(key, promise);
  return promise;
}

Expected output:

When 100 concurrent requests arrive for the same missing key, only one fetchFn call executes. The rest await the same promise.

Nginx Reverse Proxy Cache

proxy_cache_path /var/cache/nginx levels=1:2 keys_zone=mycache:10m max_size=1g;
server {
  location /api/ {
    proxy_cache mycache;
    proxy_cache_valid 200 60s;
    proxy_cache_key "$scheme$request_method$host$request_uri";
    add_header X-Cache-Status $upstream_cache_status;
    proxy_pass http://backend;
  }
}

Expected output:

First request: X-Cache-Status: MISS. Subsequent requests within 60s: X-Cache-Status: HIT. After 60s: MISS again (revalidated).

Express Response Caching Middleware

const mcache = require('memory-cache');

function cache(duration) {
  return (req, res, next) => {
    const key = '__express__' + req.originalUrl || req.url;
    const cachedBody = mcache.get(key);
    if (cachedBody) {
      res.send(cachedBody);
      return;
    }
    res.sendResponse = res.send;
    res.send = (body) => {
      mcache.put(key, body, duration * 1000);
      res.sendResponse(body);
    };
    next();
  };
}

app.get('/api/posts', cache(300), async (req, res) => {
  const posts = await db.getPosts();
  res.json(posts);
});

Expected output:

First call within 5 minutes: fetches from DB and caches. Subsequent calls: served from memory cache without DB hit.

Common Mistakes

• Caching authenticated or user-specific responses at a reverse proxy without varying by cookie or authorization header.
• Setting proxy cache sizes too small, causing frequent evictions and low hit rates.
• Not monitoring cache hit/miss ratios, so performance issues go unnoticed until a traffic spike.
• Using a single cache instance as a single point of failure — always deploy with replicas or failover.
• Caching POST responses without accounting for varying request bodies.

Practice Questions

1. How does a reverse proxy cache differ from an application-level cache?
2. What is the cache stampede problem and how does request coalescing solve it?
3. Why should you set X-Cache-Status headers in a reverse proxy?
4. How does Nginx determine the cache key for a request?
5. When would you choose Memcached over Redis for server-side caching?

Challenge

Configure Nginx as a reverse proxy cache for three different upstream services. Vary cache TTL by response status code (200: 60s, 404: 10s, 500: 0s). Add purge endpoint support using the PURGE HTTP method.

FAQ

Mini Project

Set up an Express server with an in-memory cache middleware for GET endpoints. Add a Redis backend as the primary cache with the in-memory cache as an L2 cache. Implement cache warming on startup for the top 100 most-fetched keys.

What's Next

Now explore Cache Headers to master HTTP caching directives for fine-grained control.