How to Fix Kubernetes Best-Effort Pod Eviction

In this tutorial, you'll learn about How to Fix Kubernetes Best. We cover key concepts, practical examples, and best practices.

The Problem

Your pods are evicted when the node runs low on resources, even though they have resource requests set. kubectl get pods shows Evicted status with The node was low on resource: memory. Best-Effort pods (those without resource requests/limits) are the first to be evicted under node pressure.

Quick Fix

Fix 1: Identify Best-Effort Pods

kubectl get pods -o wide
# NAME          READY   STATUS    RESTARTS   AGE
# my-app        0/1     Evicted   0          5m

kubectl get pod my-app -o yaml | grep -A 5 "resources"
# resources: {}
# (empty resources — this pod is BestEffort QoS)

Fix 2: Add Resource Requests

WRONG — pods without resource specifications:

apiVersion: v1
kind: Pod
metadata:
  name: my-app
spec:
  containers:
  - name: app
    image: nginx
    # no resources specified — BestEffort QoS

RIGHT — set at least resource requests:

apiVersion: v1
kind: Pod
metadata:
  name: my-app
spec:
  containers:
  - name: app
    image: nginx
    resources:
      requests:
        memory: "256Mi"
        cpu: "100m"
      limits:
        memory: "512Mi"
        cpu: "200m"

With requests set, the pod becomes Burstable QoS and is evicted after BestEffort pods.

Fix 3: Understand Eviction Order

# Kubernetes eviction order (worst first):
# 1. BestEffort pods (no requests/limits set)
# 2. Burstable pods using more than their requests
# 3. Burstable pods using less than their requests
# 4. Guaranteed pods (requests == limits)

To protect critical workloads, use Guaranteed QoS:

resources:
  requests:
    memory: "512Mi"
    cpu: "200m"
  limits:
    memory: "512Mi"
    cpu: "200m"

Fix 4: Add PriorityClass to Critical Pods

apiVersion: scheduling.k8s.io/v1
kind: PriorityClass
metadata:
  name: critical-app
value: 1000000
globalDefault: false
description: "Priority for production workloads"

spec:
  priorityClassName: critical-app

Higher priority pods are evicted last.

Fix 5: Set ResourceQuota with min Requests

apiVersion: v1
kind: ResourceQuota
metadata:
  name: ns-quota
spec:
  hard:
    requests.memory: "10Gi"
    limits.memory: "20Gi"
  scopeSelector:
    matchExpressions:
    - operator: In
      scopeName: PriorityClass
      values: ["system-cluster-critical"]

Fix 6: Use PodDisruptionBudget for Voluntary Disruptions

apiVersion: policy/v1
kind: PodDisruptionBudget
metadata:
  name: app-pdb
spec:
  minAvailable: 2
  selector:
    matchLabels:
      app: my-app

This does not prevent eviction but ensures a minimum number of replicas stay running.

Use DodaTech's QoS Optimizer to automatically classify pods by QoS class and recommend resource configurations that prevent unnecessary evictions.

Prevention

• Always set requests for every container (minimum CPU and memory).
• Use Guaranteed QoS for critical infrastructure pods.
• Set PriorityClass for production workloads.
• Monitor node resource pressure with kubectl describe nodes.
• Reserve system resources with --system-reserved and --kube-reserved kubelet flags.

Common Mistakes with evicted best effort

1. Using head and tail instead of pattern matching, causing runtime errors on empty lists
2. Forgetting that lazy evaluation defers computation until the value is forced, causing space leaks with unevaluated thunks
3. Using return to exit a function early instead of wrapping a pure value in the monad

These mistakes appear frequently in real-world K8S code. DodaTech's contributors have identified these patterns through analysis of open-source projects and production systems.

Practice Exercise

Write a pure function that safely divides two integers using Maybe, then test it with edge cases like division by zero and negative numbers.

This exercise reinforces the concepts covered in this guide. Try implementing it before checking online solutions.

FAQ