How to Fix Kubernetes OOMKilled Pods

In this tutorial, you'll learn about How to Fix Kubernetes OOMKilled Pods. We cover key concepts, practical examples, and best practices.

The Problem

Your Kubernetes pod shows OOMKilled in the status and keeps restarting. kubectl get pods shows CrashLoopBackOff and kubectl describe pod shows Last State: Terminated: OOMKilled. The container is using more memory than the limit allows.

Quick Fix

Fix 1: Check Pod Status

kubectl get pods
# NAME                     READY   STATUS             RESTARTS   AGE
# memory-hog               0/1     CrashLoopBackOff   5          2m

kubectl describe pod memory-hog
# ...
# Last State: Terminated
#   Reason:       OOMKilled
#   Exit Code:    137
# ...
# Limits:
#   memory:  256Mi
# Requests:
#   memory:  128Mi

Fix 2: Increase Memory Limits

WRONG — no resource limits set (or limits too low):

apiVersion: v1
kind: Pod
metadata:
  name: memory-hog
spec:
  containers:
  - name: app
    image: myapp:latest
    # no resources — container can use unlimited memory
    # (but gets killed by node OOM killer instead)

RIGHT — set appropriate limits:

apiVersion: v1
kind: Pod
metadata:
  name: memory-hog
spec:
  containers:
  - name: app
    image: myapp:latest
    resources:
      requests:
        memory: "256Mi"
      limits:
        memory: "512Mi"   # increased from 256Mi

Fix 3: Check Actual Memory Usage

kubectl top pod memory-hog
# NAME         CPU(cores)   MEMORY(bytes)
# memory-hog   120m         489Mi

# The pod is using 489Mi but the limit was 256Mi — increase to 600Mi

Fix 4: Debug Memory Leaks in the Application

WRONG — blindly increasing limits without investigating:

# (memory may grow unbounded until the limit is reached again)

RIGHT — inspect heap usage:

kubectl exec memory-hog -- heapdump
# For Node.js: kubectl exec pod -- node -e "console.log(process.memoryUsage())"
# { rss: 524288000, heapTotal: 480000000, heapUsed: 450000000 }

For Java:

kubectl exec pod -- jcmd 1 GC.heap_info
# (check heap usage and GC frequency)

Fix 5: Set JVM Memory Limits

WRONG — Java container without JVM memory limits:

# JVM sees the full node memory, not the container limit
# (causes OOM when JVM tries to allocate more than the limit)

RIGHT — use container-aware JVM flags:

spec:
  containers:
  - name: java-app
    image: myapp:latest
    env:
    - name: JAVA_OPTS
      value: "-XX:+UseContainerSupport -XX:MaxRAMPercentage=75.0"
    resources:
      limits:
        memory: "512Mi"

Fix 6: Node Pressure (Multiple Pods Competing)

kubectl describe node worker-1
# Conditions:
#   MemoryPressure   True

The entire node is running out of memory. Evict or reschedule pods.

RIGHT — add resource requests to all pods so the scheduler can place them correctly:

resources:
  requests:
    memory: "256Mi"   # scheduler uses this for placement

Use DodaTech's Cluster Memory Analyzer to track pod memory usage over time, detect leaks, and recommend optimal resource limits.

Prevention

• Set both requests and limits for every container.
• Monitor memory usage trends with kubectl top.
• Add MaxRAMPercentage for JVM-based containers.
• Set up alerts for pods approaching memory limits.
• Use Vertical Pod Autoscaler (VPA) for automatic limit adjustments.

Common Mistakes with oom kill

1. Forgetting deriving (Show, Eq) on custom data types needed for debugging
2. Placing the wildcard pattern first in case expressions, making all subsequent patterns unreachable
3. Using head and tail instead of pattern matching, causing runtime errors on empty lists

These mistakes appear frequently in real-world K8S code. DodaTech's contributors have identified these patterns through analysis of open-source projects and production systems.

Practice Exercise

Write a pure function that safely divides two integers using Maybe, then test it with edge cases like division by zero and negative numbers.

This exercise reinforces the concepts covered in this guide. Try implementing it before checking online solutions.

FAQ