Circuit Breaker Project — Complete Implementation Guide
In this tutorial, you will learn about Circuit Breaker Project. We cover key concepts, practical examples, and best practices to help you master this topic.
This circuit breaker project guides you through building a complete resilience layer for a microservice that depends on HTTP APIs, a database, and a message queue.
What You'll Learn
By the end of this project, you will have built a full circuit breaker system with per-service configuration, event-driven monitoring, fallback responses, and a health dashboard.
Why It Matters
A complete circuit breaker project ties together everything you've learned into a single deployable service. This is the difference between knowing the theory and being able to implement it in production.
Real-World Use
DodaTech uses this exact project structure for every new microservice. The circuit breaker layer is added during the first sprint and tuned throughout the service's lifecycle.
Project Learning Path
flowchart LR
A[Spring Boot Circuit Breaker] --> B[Circuit Breaker Project]
B --> C[Architecture]
B --> D[Implementation]
B --> E[Testing]
B --> F{You Are Here}
style F fill:#f90,color:#fff
Project Architecture
The project consists of a backend service with three downstream dependencies, each protected by a circuit breaker.
class ServiceArchitecture {
constructor() {
this.services = {
"payment-api": {
type: "http",
threshold: 5,
timeout: 10000,
fallback: "use-cached-payment-methods"
},
"user-database": {
type: "database",
threshold: 3,
timeout: 5000,
fallback: "use-local-cache"
},
"notification-queue": {
type: "queue",
threshold: 10,
timeout: 30000,
fallback: "store-for-retry"
}
};
}
getConfig(serviceName) {
return this.services[serviceName];
}
}
const arch = new ServiceArchitecture();
console.log("Payment API config:", arch.getConfig("payment-api"));
console.log("Database config:", arch.getConfig("user-database"));
// Payment API config: { type: 'http', threshold: 5, ... }
// Database config: { type: 'database', threshold: 3, ... }
Base Circuit Breaker Class
Start with a reusable base circuit breaker that all service-specific circuits extend.
class BaseCircuitBreaker {
constructor(name, options = {}) {
this.name = name;
this.threshold = options.threshold || 5;
this.resetTimeout = options.resetTimeout || 30000;
this.state = "closed";
this.failureCount = 0;
this.nextAttempt = Date.now();
this.metrics = { opens: 0, closes: 0, fallbacks: 0, totalCalls: 0 };
this.hooks = { onOpen: [], onClose: [], onHalfOpen: [], onFallback: [] };
}
on(event, callback) {
if (this.hooks[event]) {
this.hooks[event].push(callback);
}
}
async call(fn, fallbackFn) {
this.metrics.totalCalls++;
if (this.state === "open") {
if (Date.now() >= this.nextAttempt) {
this.transitionTo("half-open");
} else {
return this.useFallback(fallbackFn);
}
}
try {
const result = await fn();
this.onSuccess();
return result;
} catch (err) {
this.onFailure(err);
return this.useFallback(fallbackFn);
}
}
onSuccess() {
this.failureCount = 0;
if (this.state === "half-open") {
this.transitionTo("closed");
}
}
onFailure(err) {
this.failureCount++;
if (this.state === "half-open" || this.failureCount >= this.threshold) {
this.transitionTo("open");
}
}
transitionTo(newState) {
const oldState = this.state;
this.state = newState;
const event = { from: oldState, to: newState, time: new Date(), name: this.name };
if (newState === "open") {
this.metrics.opens++;
this.nextAttempt = Date.now() + this.resetTimeout;
this.hooks.onOpen.forEach(h => h(event));
} else if (newState === "closed") {
this.metrics.closes++;
this.hooks.onClose.forEach(h => h(event));
} else if (newState === "half-open") {
this.hooks.onHalfOpen.forEach(h => h(event));
}
console.log(`[${this.name}] ${oldState} -> ${newState}`);
}
async useFallback(fallbackFn) {
if (fallbackFn) {
this.metrics.fallbacks++;
this.hooks.onFallback.forEach(h => h({ name: this.name, time: new Date() }));
return fallbackFn();
}
throw new Error(`Circuit ${this.name} is open and no fallback provided`);
}
getStatus() {
return {
name: this.name,
state: this.state,
failureCount: this.failureCount,
metrics: { ...this.metrics }
};
}
}
const cb = new BaseCircuitBreaker("payment-api", { threshold: 3, resetTimeout: 5000 });
console.log("Created:", cb.getStatus());
// Created: { name: 'payment-api', state: 'closed', failureCount: 0, metrics: { opens: 0, ... } }
HTTP Service Circuit Breaker
Extend the base class with HTTP-specific error classification and response handling.
class HttpCircuitBreaker extends BaseCircuitBreaker {
constructor(name, baseUrl, options = {}) {
super(name, options);
this.baseUrl = baseUrl;
}
async request(path, options = {}) {
const url = `${this.baseUrl}${path}`;
return this.call(
async () => {
const response = await fetch(url, {
...options,
signal: AbortSignal.timeout(options.timeout || 10000)
});
if (!response.ok && this.isServerError(response.status)) {
throw new HttpError(response.status, response.statusText);
}
return response.json();
},
() => ({ cached: true, message: "Fallback response" })
);
}
isServerError(status) {
return status >= 500 || status === 429;
}
}
class HttpError extends Error {
constructor(status, message) {
super(message);
this.status = status;
}
}
const api = new HttpCircuitBreaker("payment-api", "https://api.payments.com", {
threshold: 3,
resetTimeout: 10000
});
api.on("open", e => console.log("ALERT: HTTP circuit opened", e.name));
Database Circuit Breaker
Extend the base class for database operations with connection pooling protection.
class DatabaseCircuitBreaker extends BaseCircuitBreaker {
constructor(name, pool, options = {}) {
super(name, { threshold: 3, resetTimeout: 60000, ...options });
this.pool = pool;
}
async query(sql, params = []) {
return this.call(
async () => {
const client = await this.pool.connect();
try {
const result = await client.query(sql, params);
return result.rows;
} finally {
client.release();
}
},
() => {
console.log(`[${this.name}] Using cached query result`);
return { cached: true, rows: [] };
}
);
}
}
const { Pool } = require("pg");
const pool = new Pool({ max: 10 });
const db = new DatabaseCircuitBreaker("user-db", pool);
Monitoring Dashboard
Build a health endpoint that shows the status of all circuit breakers.
class CircuitMonitor {
constructor() {
this.circuits = [];
}
register(circuit) {
this.circuits.push(circuit);
}
getDashboard() {
return {
timestamp: new Date().toISOString(),
summary: {
total: this.circuits.length,
closed: this.circuits.filter(c => c.state === "closed").length,
open: this.circuits.filter(c => c.state === "open").length,
halfOpen: this.circuits.filter(c => c.state === "half-open").length
},
circuits: this.circuits.map(c => c.getStatus())
};
}
getAlertSummary() {
const openCircuits = this.circuits.filter(c => c.state === "open");
if (openCircuits.length > 0) {
return {
level: "warning",
message: `${openCircuits.length} circuit(s) are open`,
circuits: openCircuits.map(c => c.name)
};
}
return { level: "ok", message: "All circuits closed" };
}
}
const monitor = new CircuitMonitor();
monitor.register(api);
monitor.register(db);
console.log("Dashboard:", JSON.stringify(monitor.getDashboard(), null, 2));
console.log("Alert:", monitor.getAlertSummary());
// Dashboard: { timestamp: "...", summary: { total: 2, closed: 2, ... }, circuits: [...] }
// Alert: { level: "ok", message: "All circuits closed" }
Testing the Complete System
Integration test that exercises all circuit breakers with failure injection.
async function runIntegrationTest() {
const api = new HttpCircuitBreaker("test-api", "http://localhost:9999", {
threshold: 2,
resetTimeout: 1000
});
const monitor = new CircuitMonitor();
monitor.register(api);
for (let i = 0; i < 5; i++) {
try {
await api.request("/test");
} catch (err) {
console.log("Request", i + 1, ":", err.message);
}
}
console.log("Final status:", api.getStatus());
console.log("Dashboard:", monitor.getDashboard().summary);
}
runIntegrationTest();
// Request 1: fetch failed
// Request 2: fetch failed
// Request 3: Circuit test-api is open and no fallback provided
// Request 4: Circuit test-api is open and no fallback provided
// Request 5: Circuit test-api is open and no fallback provided
// Final status: { name: 'test-api', state: 'open', failureCount: 2, ... }
// Dashboard: { total: 1, closed: 0, open: 1, halfOpen: 0 }
Common Mistakes
Not separating circuit breaker instances per service -- Sharing one circuit breaker for all downstream services means one failing service blocks all others.
Forgetting to register circuits with the monitor -- The monitoring dashboard only shows registered circuits. Always register new circuit breakers on creation.
Using the same timeout for all services -- HTTP APIs typically need 10 seconds, databases 5 seconds, and queues 30 seconds. Match the timeout to the service's expected response time.
Not handling the case where all fallbacks fail -- Fallback functions can fail too. Wrap fallback calls in try-catch and return a final default value.
Skipping the half-open test with real services -- Always test that the half-open probe actually reaches the real service and doesn't get intercepted by middleware or load balancers.
Practice Questions
What are the three main service types in this project? HTTP API, database, and message queue. Each has different threshold and timeout requirements.
How does the monitor's alert summary help operators? It provides a quick overview of how many circuits are open, which services are affected, and whether manual intervention is needed.
Why does the database circuit breaker have a longer reset timeout than the HTTP one? Database failures often indicate infrastructure issues (server restart, Network Partition) that take longer to resolve than transient HTTP failures.
Challenge: Extend the project to support dynamic threshold adjustment based on historical failure rates.
class AdaptiveCircuitBreaker extends BaseCircuitBreaker {
constructor(name, options = {}) {
super(name, options);
this.historyWindow = [];
this.adaptInterval = options.adaptInterval || 60000;
this.lastAdapt = Date.now();
}
recordResult(success) {
this.historyWindow.push({ success, time: Date.now() });
this.cleanupHistory();
if (Date.now() - this.lastAdapt > this.adaptInterval) {
this.adaptThreshold();
this.lastAdapt = Date.now();
}
}
adaptThreshold() {
const rate = this.calculateFailureRate();
if (rate > 0.7) {
this.threshold = Math.max(2, this.threshold - 1);
console.log(`Adapting threshold down to ${this.threshold}`);
} else if (rate < 0.2) {
this.threshold = Math.min(20, this.threshold + 1);
console.log(`Adapting threshold up to ${this.threshold}`);
}
}
calculateFailureRate() {
if (this.historyWindow.length < 10) return 0;
const failures = this.historyWindow.filter(r => !r.success).length;
return failures / this.historyWindow.length;
}
cleanupHistory() {
const cutoff = Date.now() - 300000;
this.historyWindow = this.historyWindow.filter(r => r.time > cutoff);
}
}
FAQ
Mini Project
The project is complete. Extend it by adding a Grafana dashboard that displays real-time circuit breaker states, failure rates, and alert history using the metrics collected by the monitor.
What's Next
Congratulations on completing the circuit breaker project. Next, learn graceful shutdown patterns to ensure your services shut down cleanly without dropping requests.
Built by the developers of DodaTech
Doda Browser, DodaZIP & Durga Antivirus Pro