Graceful Shutdown Project — Complete Implementation Guide

DodaTech Updated 2026-06-28 8 min read

In this tutorial, you will learn about Graceful Shutdown Project. We cover key concepts, practical examples, and best practices to help you master this topic.

This graceful shutdown project guides you through building a complete shutdown system for a production Express.js application with Kubernetes deployment support.

What You'll Learn

By the end of this project, you will have built a full graceful shutdown implementation that handles signals, drains connections, closes database pools, manages health checks, and integrates with Kubernetes.

Why It Matters

A complete graceful shutdown project ties together signal handling, connection draining, resource cleanup, and Orchestration integration into a single deployable system.

Real-World Use

DodaTech uses this exact project structure as the template for every new Node.js microservice. The shutdown module is added during the initial scaffolding and remains consistent across all services.

Project Learning Path

flowchart LR
  A[Express Graceful Shutdown] --> B[Graceful Shutdown Project]
  B --> C[Architecture]
  B --> D[Implementation]
  B --> E[Testing]
  B --> F{You Are Here}
  style F fill:#f90,color:#fff

Project Architecture

The project consists of a ShutdownManager that coordinates all shutdown phases across multiple components.

class ProjectArchitecture {
  constructor() {
    this.components = {
      httpServer: { type: "server", drainOrder: 1, requires: [] },
      healthServer: { type: "server", drainOrder: 0, requires: [] },
      databasePool: { type: "resource", drainOrder: 3, requires: ["httpServer"] },
      messageQueue: { type: "resource", drainOrder: 4, requires: ["httpServer"] },
      cacheClient: { type: "resource", drainOrder: 5, requires: [] },
      logFlusher: { type: "resource", drainOrder: 6, requires: [] }
    };
  }

  getDrainOrder() {
    return Object.entries(this.components)
      .sort(([, a], [, b]) => a.drainOrder - b.drainOrder)
      .map(([name]) => name);
  }

  validate() {
    console.log("Validating shutdown architecture...");
    const order = this.getDrainOrder();
    console.log("Drain order:", order.join(" -> "));
    return { valid: true, order };
  }
}

const arch = new ProjectArchitecture();
arch.validate();
// Validating shutdown architecture...
// Drain order: healthServer -> httpServer -> databasePool -> messageQueue -> cacheClient -> logFlusher

Shutdown Manager

The core ShutdownManager class coordinates all phases.

class ShutdownManager {
  constructor(options = {}) {
    this.timeout = options.timeout || 25000;
    this.phases = [];
    this.state = "running";
  }

  addPhase(name, handler) {
    this.phases.push({ name, handler });
  }

  async shutdown(signal) {
    console.log(`[${new Date().toISOString()}] Shutdown via ${signal}`);
    this.state = "shutting-down";
    const startTime = Date.now();

    for (const phase of this.phases) {
      const remaining = this.timeout - (Date.now() - startTime);
      if (remaining <= 0) {
        console.log(`Timeout exceeded, skipping phase: ${phase.name}`);
        break;
      }

      try {
        console.log(`Phase: ${phase.name}`);
        await Promise.race([
          phase.handler(),
          new Promise((_, reject) =>
            setTimeout(() => reject(new Error(`${phase.name} timeout`)), remaining)
          )
        ]);
        console.log(`Phase complete: ${phase.name}`);
      } catch (err) {
        console.error(`Phase failed: ${phase.name} - ${err.message}`);
      }
    }

    const totalTime = Date.now() - startTime;
    console.log(`Shutdown completed in ${totalTime}ms`);
    process.exit(0);
  }

  getStatus() {
    return {
      state: this.state,
      phaseCount: this.phases.length
    };
  }
}

const manager = new ShutdownManager({ timeout: 20000 });
manager.addPhase("health-unhealthy", () => Promise.resolve());
manager.addPhase("http-drain", () => Promise.resolve());
manager.addPhase("db-close", () => Promise.resolve());
manager.shutdown("SIGTERM");
// [2026-06-28T12:00:00.000Z] Shutdown via SIGTERM
// Phase: health-unhealthy
// Phase complete: health-unhealthy
// Phase: http-drain
// Phase complete: http-drain
// Phase: db-close
// Phase complete: db-close
// Shutdown completed in 0ms

Health Check Integration

Separate health check server that marks unhealthy immediately on shutdown.

const http = require("http");

class HealthEndpoint {
  constructor() {
    this.ready = true;
    this.server = http.createServer((req, res) => {
      if (req.url === "/healthz") {
        res.writeHead(200);
        res.end("ok");
      } else if (req.url === "/readyz") {
        res.writeHead(this.ready ? 200 : 503);
        res.end(this.ready ? "ready" : "not ready");
      } else {
        res.writeHead(404);
        res.end();
      }
    });
  }

  start(port = 8081) {
    this.server.listen(port, () => {
      console.log(`Health endpoints on :${port}/healthz and /readyz`);
    });
  }

  async markNotReady() {
    this.ready = false;
    console.log("Readiness endpoint -> 503");
    await new Promise(r => setTimeout(r, 3000));
    console.log("Drain delay complete");
  }

  async close() {
    return new Promise(r => this.server.close(r));
  }
}

const health = new HealthEndpoint();
health.start();
health.markNotReady().then(() => health.close());
// Health endpoints on :8081/healthz and /readyz
// Readiness endpoint -> 503
// Drain delay complete

HTTP Server with Connection Draining

Express server with connection tracking and drain timeout.

const express = require("express");
const http = require("http");

class DrainingHttpServer {
  constructor() {
    this.app = express();
    this.server = http.createServer(this.app);
    this.connections = new Set();
    this.requests = new Map();
    this.requestCounter = 0;
    this.setupTracking();
  }

  setupTracking() {
    this.server.on("connection", (socket) => {
      this.connections.add(socket);
      socket.on("close", () => this.connections.delete(socket));
    });

    this.app.use((req, res, next) => {
      const id = ++this.requestCounter;
      this.requests.set(id, { url: req.url, start: Date.now() });
      res.on("finish", () => this.requests.delete(id));
      next();
    });
  }

  async drain(timeoutMs = 10000) {
    console.log(`Draining ${this.requests.size} active requests`);
    const start = Date.now();

    return new Promise((resolve) => {
      const check = () => {
        const elapsed = Date.now() - start;
        if (this.requests.size === 0) {
          console.log("All requests drained");
          resolve();
        } else if (elapsed > timeoutMs) {
          console.log(`Drain timeout with ${this.requests.size} requests remaining`);
          resolve();
        } else {
          setTimeout(check, 200);
        }
      };
      check();
    });
  }

  async close() {
    return new Promise(r => this.server.close(r));
  }
}

const httpServer = new DrainingHttpServer();

Database Pool Manager

PostgreSQL pool with drain capabilities.

const { Pool } = require("pg");

class DrainingDatabasePool {
  constructor(config) {
    this.pool = new Pool(config);
    this.activeQueries = 0;
  }

  async query(text, params) {
    this.activeQueries++;
    try {
      return await this.pool.query(text, params);
    } finally {
      this.activeQueries--;
    }
  }

  async drain(timeoutMs = 5000) {
    console.log(`Draining ${this.activeQueries} active queries`);
    const start = Date.now();

    while (this.activeQueries > 0) {
      if (Date.now() - start > timeoutMs) break;
      await new Promise(r => setTimeout(r, 200));
    }

    await this.pool.end();
    console.log("Database pool closed");
  }
}

const db = new DrainingDatabasePool({
  connectionString: "postgresql://localhost/mydb",
  max: 10
});

Complete Shutdown Sequence

Tie all components together.

class ProductionShutdown {
  constructor() {
    this.manager = new ShutdownManager({ timeout: 25000 });
    this.health = new HealthEndpoint();
    this.http = new DrainingHttpServer();
    this.db = new DrainingDatabasePool({ connectionString: "postgresql://localhost/mydb" });
    this.setupPhases();
  }

  setupPhases() {
    this.manager.addPhase("mark-unhealthy", () => this.health.markNotReady());
    this.manager.addPhase("drain-http", () => this.http.drain(10000));
    this.manager.addPhase("close-http", () => this.http.close());
    this.manager.addPhase("drain-db", () => this.db.drain(5000));
    this.manager.addPhase("close-health", () => this.health.close());
  }

  start() {
    this.health.start(8081);
    this.http.server.listen(3000);

    process.on("SIGTERM", (signal) => this.manager.shutdown(signal));
    process.on("SIGINT", (signal) => this.manager.shutdown(signal));

    console.log("Production server started");
  }
}

const app = new ProductionShutdown();
app.start();

Kubernetes Deployment Config

Deployment YAML with preStop hook and appropriate termination grace period.

class DeploymentConfigGenerator {
  static generate(appName, imageTag, replicas = 3) {
    return {
      apiVersion: "apps/v1",
      kind: "Deployment",
      metadata: { name: appName },
      spec: {
        replicas,
        selector: { matchLabels: { app: appName } },
        template: {
          metadata: { labels: { app: appName } },
          spec: {
            terminationGracePeriodSeconds: 40,
            containers: [{
              name: appName,
              image: `registry.dodatech.com/${appName}:${imageTag}`,
              ports: [
                { containerPort: 3000, name: "http" },
                { containerPort: 8081, name: "health" }
              ],
              readinessProbe: {
                httpGet: { path: "/readyz", port: 8081 },
                initialDelaySeconds: 5,
                periodSeconds: 5
              },
              livenessProbe: {
                httpGet: { path: "/healthz", port: 8081 },
                initialDelaySeconds: 15,
                periodSeconds: 10
              },
              lifecycle: {
                preStop: {
                  exec: {
                    command: [
                      "/bin/sh", "-c",
                      "curl -X POST http://localhost:8081/drain && sleep 5"
                    ]
                  }
                }
              }
            }]
          }
        }
      }
    };
  }
}

const config = DeploymentConfigGenerator.generate("my-service", "v2.1.0", 5);
console.log("Generated deployment with preStop hook");

Common Mistakes

Not setting terminationGracePeriodSeconds high enough -- The default 30 seconds is often too short for applications with long-running requests. Set it to maxRequestDuration + drainTime + buffer.
Putting sleep in preStop instead of triggering application drain -- The preStop should call the application's shutdown endpoint, not just wait. The application handles timing internally.
Closing the health server before the main server -- Health checks should remain available until the main server finishes draining. Close health server last.
Not handling SIGTERM in all worker processes -- In cluster mode, each worker needs its own signal handlers. The primary should forward SIGTERM to workers.
Testing shutdown only in development -- Shutdown behavior differs in Kubernetes. Always test the full Kubernetes termination sequence in a staging environment.

Practice Questions

What is the correct order of shutdown phases in this project? Mark health unhealthy -> drain HTTP requests -> close HTTP server -> drain database -> close health server.
Why does the health server stay open while the main server drains? So Kubernetes can continue checking health during the drain. The readiness probe returns unhealthy but the endpoint remains available.
How does the preStop hook interact with the application's shutdown? The preStop hook triggers the application's shutdown preparation (marking unhealthy), then sleeps to allow endpoint propagation before SIGTERM arrives.
Challenge: Extend the project to support graceful shutdown with multiple database pools and message queue consumers.

class ExtendedShutdown extends ProductionShutdown {
  constructor() {
    super();
    this.queues = [];
  }

  addQueue(name, consumer) {
    this.queues.push({ name, consumer });
    this.manager.addPhase(`drain-queue-${name}`, () => consumer.drain());
  }

  getShutdownPlan() {
    return this.manager.phases.map(p => p.name);
  }
}

const ext = new ExtendedShutdown();
ext.addQueue("notifications", { drain: () => Promise.resolve() });
ext.addQueue("file-processing", { drain: () => Promise.resolve() });
console.log("Shutdown plan:", ext.getShutdownPlan());

FAQ

How do I test this project in a local Kubernetes cluster?

Use minikube or kind. Deploy the Docker image, send traffic with a load tester, then delete the pod and observe the shutdown sequence.

What happens if the shutdown times out?

The process exits regardless. Kubernetes then sends SIGKILL. Any remaining connections are orphaned and cleaned up by the respective systems.

Should I use a library for graceful shutdown?

Libraries like http-shutdown and stoppable simplify the HTTP server part but don't handle application-specific resources. A custom implementation gives more control.

How do I handle stateful services like databases?

Databases should be closed last. Active transactions roll back, and the pool releases connections. The database server handles cleanup if connections are lost.

Can I reuse this project for non-HTTP services?

Yes. Replace the HTTP server with your protocol's server (gRPC, WebSocket, TCP). The shutdown manager pattern remains the same.

Mini Project

The project is complete. Deploy it to a Kubernetes cluster and verify graceful shutdown by:

Sending continuous traffic with a load tester
Deleting the pod with kubectl delete pod
Verifying zero failed requests during the termination

What's Next

Congratulations on completing the graceful shutdown project. Next, learn health check endpoints to complete your understanding of production readiness.

← Previous Graceful Shutdown FastAPI Deep Dive Next → Graceful Shutdown Django Deep Dive

Built by the developers of DodaTech

Doda Browser, DodaZIP & Durga Antivirus Pro

Home Browse Graceful Shutdown