Retry Best Practices — Comprehensive Implementation Guide
In this tutorial, you will learn about Retry Best Practices. We cover key concepts, practical examples, and best practices to help you master this topic.
Retry best practices guide you in designing resilient systems that handle failures gracefully, using appropriate retry strategies without causing cascading failures or wasting resources.
What You'll Learn
By the end of this tutorial, you will know the industry-standard best practices for implementing retries, common pitfalls to avoid, and how to design for resilience.
Why It Matters
Retries are powerful but dangerous. Done wrong, they amplify failures. DodaTech follows these best practices to ensure retries improve reliability without causing harm.
Real-World Use
DodaTech's architecture review checklist includes 15 retry best practices that every service must follow before deployment.
Best Practices Learning Path
flowchart LR
A[Retry Monitoring] --> B[Best Practices]
B --> C[Do's]
B --> D[Don'ts]
B --> E{You Are Here}
style E fill:#f90,color:#fff
Best Practice 1: Only Retry Transient Failures
The most important rule: only retry failures that have a chance of succeeding on subsequent attempts.
function isTransientError(err) {
// Network errors
if (err.code === "ECONNRESET" || err.code === "ETIMEDOUT") return true;
// HTTP 5xx and 429
if (err.status === 429 || (err.status >= 500 && err.status < 600)) return true;
// Database deadlocks
if (err.code === "40001" || err.code === "40P01") return true;
return false;
}
async function smartRetry(fn) {
for (let attempt = 0; attempt < 3; attempt++) {
try {
return await fn();
} catch (err) {
if (!isTransientError(err)) throw err;
if (attempt === 2) throw err;
await backoff(attempt);
}
}
}
Best Practice 2: Use Exponential Backoff with Jitter
Always use exponential backoff with jitter. Never use fixed-interval retries in production.
function bestBackoff(attempt, baseDelay = 200, maxDelay = 30000) {
const exponential = Math.min(baseDelay * Math.pow(2, attempt), maxDelay);
// Full jitter: randomize between 0 and exponential
return Math.random() * exponential;
}
Best Practice 3: Set a Maximum Retry Budget
Limit the total time or total attempts for retries to prevent runaway retry loops.
class RetryBudget {
constructor(maxRetries = 3, maxDuration = 30000) {
this.maxRetries = maxRetries;
this.maxDuration = maxDuration;
}
async execute(fn) {
const start = Date.now();
for (let attempt = 0; attempt < this.maxRetries; attempt++) {
if (Date.now() - start > this.maxDuration) {
throw new Error("Retry budget exceeded");
}
try {
return await fn();
} catch (err) {
if (attempt === this.maxRetries - 1) throw err;
await new Promise(r => setTimeout(r, 200 * Math.pow(2, attempt)));
}
}
}
}
Best Practice 4: Combine with Circuit Breakers
Retries handle transient failures. Circuit breakers prevent retries from overwhelming a failing service.
class RetryWithCircuitBreaker {
constructor(options) {
this.retryCount = options.retryCount || 3;
this.failureThreshold = options.failureThreshold || 5;
this.resetTimeout = options.resetTimeout || 30000;
this.failures = 0;
this.state = "closed";
this.nextAttempt = Date.now();
}
async call(fn) {
if (this.state === "open") {
if (Date.now() > this.nextAttempt) {
this.state = "half-open";
} else {
throw new Error("Circuit breaker open");
}
}
for (let i = 0; i < this.retryCount; i++) {
try {
const result = await fn();
this.onSuccess();
return result;
} catch (err) {
if (i === this.retryCount - 1) {
this.onFailure();
throw err;
}
await new Promise(r => setTimeout(r, 200 * Math.pow(2, i)));
}
}
}
onSuccess() {
this.failures = 0;
this.state = "closed";
}
onFailure() {
this.failures++;
if (this.failures >= this.failureThreshold) {
this.state = "open";
this.nextAttempt = Date.now() + this.resetTimeout;
}
}
}
Best Practice 5: Make Operations Idempotent
Design all operations that may be retried to be idempotent. Use idempotency keys for non-idempotent operations.
// Non-idempotent (BAD for retry):
app.post("/charge", async (req, res) => {
await chargeCustomer(req.body.amount); // May double-charge on retry
});
// Idempotent (GOOD for retry):
app.post("/charge", async (req, res) => {
const key = req.headers["idempotency-key"];
if (await hasBeenProcessed(key)) {
return res.json(await getResult(key));
}
await chargeCustomer(req.body.amount);
await storeResult(key, { success: true });
});
Best Practice 6: Set Total Timeouts
Each retry attempt should have a timeout, and the entire retry sequence should have a total timeout.
async function retryWithTimeout(fn, options = {}) {
const attemptTimeout = options.attemptTimeout || 5000;
const totalTimeout = options.totalTimeout || 30000;
const start = Date.now();
for (let attempt = 0; attempt < 3; attempt++) {
if (Date.now() - start > totalTimeout) {
throw new Error("Total retry timeout exceeded");
}
try {
return await Promise.race([
fn(),
new Promise((_, reject) =>
setTimeout(() => reject(new Error("Attempt timeout")), attemptTimeout)
)
]);
} catch (err) {
if (attempt === 2) throw err;
await sleep(200 * Math.pow(2, attempt));
}
}
}
Common Mistakes Summary
- Retrying non-idempotent operations -- Creates duplicates
- Retrying without backoff -- Creates retry storms
- No maximum retry limit -- Infinite retry loops
- Retrying 4xx errors -- Will never succeed
- No monitoring -- Cannot detect retry problems
- No circuit breaker -- Retries amplify failures
- Same policy for all operations -- Different needs
- No jitter -- Thundering herd
Practice Questions
What is the most important retry best practice? Only retry transient failures. Non-transient errors will never succeed and waste resources.
Why combine retries with circuit breakers? Retries handle short-lived failures. Circuit breakers stop retries when a service is clearly down.
What is a retry budget? A limit on total retry time or attempts, preventing runaway retry loops from exhausting resources.
Challenge: Create a checklist for reviewing retry implementations.
Retry Review Checklist:
- [ ] Only transient errors are retried
- [ ] Exponential backoff with jitter
- [ ] Maximum retry count set (3-5)
- [ ] Total timeout configured
- [ ] Circuit breaker integrated
- [ ] Operations are idempotent
- [ ] Retry metrics and monitoring
- [ ] Non-retryable errors documented
FAQ
Mini Project
Build a reference retry implementation that follows all best practices.
class BestPracticeRetry {
constructor(options = {}) {
this.maxRetries = options.maxRetries || 3;
this.baseDelay = options.baseDelay || 200;
this.maxDelay = options.maxDelay || 30000;
this.totalTimeout = options.totalTimeout || 30000;
this.attemptTimeout = options.attemptTimeout || 10000;
}
async execute(fn, options = {}) {
const isTransient = options.isTransient ||
((err) => err.code === "ECONNRESET" || err.status === 429 || err.status >= 500);
const start = Date.now();
for (let attempt = 0; attempt < this.maxRetries; attempt++) {
if (Date.now() - start > this.totalTimeout) {
throw new Error("Retry budget exceeded");
}
try {
const result = await Promise.race([
fn(),
new Promise((_, reject) =>
setTimeout(() => reject(Object.assign(new Error("Timeout"), { code: "TIMEOUT" })), this.attemptTimeout)
)
]);
return result;
} catch (err) {
if (!isTransient(err)) throw err;
if (attempt === this.maxRetries - 1) throw err;
const delay = Math.random() * Math.min(this.baseDelay * Math.pow(2, attempt), this.maxDelay);
await new Promise(r => setTimeout(r, delay));
}
}
}
}
What's Next
Now that you understand retry best practices, explore advanced retry patterns. Then apply everything in the comprehensive retry project.
Built by the developers of DodaTech
Doda Browser, DodaZIP & Durga Antivirus Pro