How to Fix Kubernetes Pod Evicted Status
In this tutorial, you'll learn about How to Fix Kubernetes Pod Evicted Status. We cover key concepts, practical examples, and best practices.
The Problem
A pod shows Status: Evicted after kubectl get pods and the application becomes unavailable because the kubelet evicted the pod due to node pressure (disk, memory, or PID).
Quick Fix
Describe the Evicted Pod to Find the Reason
kubectl describe pod <pod-name>
# ...
# Status: Failed
# Reason: Evicted
# Message: The node was low on resource: ephemeral-storage.
kubectl describe pod shows the eviction reason. Common reasons are NodeDiskPressure, NodeMemoryPressure, and NodePIDPressure.
Delete the Evicted Pod
kubectl delete pod <pod-name> --grace-period=0 --force
# pod "<pod-name>" deleted
Evicted pods remain in the list until deleted. If the pod is managed by a Deployment or StatefulSet, the controller recreates it automatically after you delete the evicted instance.
Check Node Resource Usage
kubectl top nodes
# NAME CPU(cores) CPU% MEMORY(bytes) MEMORY%
# node-01 450m 22% 2048Mi 65%
kubectl describe node node-01 | grep -A 5 Conditions
# DiskPressure False
# MemoryPressure False
# PIDPressure False
Use kubectl top nodes to see current utilization and kubectl describe node to check pressure conditions. If pressure is True, the node needs more resources or fewer pods.
Set Resource Requests and Limits
node -e "
const pod = {
spec: {
containers: [{
name: 'app',
resources: {
requests: { memory: '256Mi', cpu: '250m' },
limits: { memory: '512Mi', cpu: '500m' }
}
}]
}
};
console.log('Resource requests and limits configured');
"
# Resource requests and limits configured
Always set resources.requests and resources.limits in your pod specs. Without requests, the scheduler may overcommit the node and trigger evictions.
Drain and Cordon a Problematic Node
kubectl cordon node-01
# node/node-01 cordoned
kubectl drain node-01 --ignore-daemonsets --delete-emptydir-data
# node/node-01 drained
If a node experiences repeated pressure, cordon it to prevent new pod assignments and drain it to move existing workloads. Investigate the root cause before uncordoning.
Use kubectl describe for Detailed Diagnostics
kubectl describe pod <pod-name>
# Events:
# Type Reason Age From Message
# ---- ------ ---- ---- -------
# Warning BackOff 5m kubelet Back-off restarting failed container
The Events section at the bottom of kubectl describe output is the most valuable diagnostic tool. It shows a chronological log of scheduling failures, image pull errors, and container crashes.
Additional Troubleshooting
# Check the error message and stack trace for more context
echo "Review the full error output to identify the root cause"
If the above steps do not resolve the issue, examine the complete error message and stack trace. Often the key detail is in the middle of the traceback rather than the final line. Search for the error message in the project documentation or issue tracker for additional solutions.
Prevention
- Always set resource requests and limits on every container
- Monitor node pressure conditions with
kubectl describe noderegularly - Use
kubectl taintto cordon nodes that are at capacity - Set
eviction-hardandeviction-softkubelet thresholds to appropriate levels for your workload
Built by the developers of DodaTech
Doda Browser, DodaZIP & Durga Antivirus Pro