Skip to content

AWS EC2 Spot Instance Termination Fix

DodaTech Updated 2026-06-24 2 min read

In this tutorial, you'll learn about AWS EC2 Spot Instance Termination Fix. We cover key concepts, practical examples, and best practices to help you understand and apply this topic effectively.

Your EC2 Spot Instance terminates unexpectedly with instance-terminated-by-service — AWS reclaimed the capacity when the Spot price exceeded your bid or demand increased.

Step-by-Step Fix

1. Listen for the termination notice

# Check for the termination notice metadata endpoint (2-minute warning)
curl http://169.254.169.254/latest/meta-data/spot/termination-time

Expected output (if termination is imminent):

2024-01-15T10:02:00Z

If no termination is pending, the request hangs or returns empty.

2. Handle termination in application code

import requests
import time
import signal

def check_termination():
    try:
        resp = requests.get(
            'http://169.254.169.254/latest/meta-data/spot/termination-time',
            timeout=5
        )
        if resp.status_code == 200:
            print("Termination imminent, saving state...")
            # Save checkpoint, drain connections, upload logs
            save_checkpoint()
            return True
    except requests.exceptions.Timeout:
        pass
    return False

# Poll for termination notice every 5 seconds
while True:
    if check_termination():
        graceful_shutdown()
        break
    time.sleep(5)

3. Use a Spot Fleet with diversified instance types

# Wrong: single instance type bid
aws ec2 request-spot-instances \
  --instance-count 1 \
  --type persistent \
  --instance-type c5.large

# Right: Spot Fleet with diversified types and allocation strategy
aws ec2 request-spot-fleet \
  --spot-fleet-request-config file://spot-fleet-config.json

// spot-fleet-config.json
{
    "AllocationStrategy": "diversified",
    "TargetCapacity": 5,
    "SpotPrice": "0.10",
    "LaunchTemplateConfigs": [
        {
            "LaunchTemplateSpecification": {
                "LaunchTemplateId": "lt-0123456789",
                "Version": "1]
            },
            "Overrides": [
                {"InstanceType": "c5.large", "SubnetId": "subnet-abc"},
                {"InstanceType": "c5a.large", "SubnetId": "subnet-abc"},
                {"InstanceType": "c5d.large", "SubnetId": "subnet-def"},
                {"InstanceType": "m5.large", "SubnetId": "subnet-def"}
            ]
        }
    ]
}

4. Use mixed instances with on-demand fallback

aws ec2 request-spot-fleet \
  --spot-fleet-request-config '{
    "TargetCapacity": 10,
    "OnDemandTargetCapacity": 2,
    "SpotTargetCapacity": 8,
    "AllocationStrategy": "lowestPrice",
    "LaunchTemplateConfigs": [...]
  }'

Prevention

  • Always write your application to handle Spot termination gracefully using the termination notice endpoint.
  • Use Spot Fleet or EC2 Fleet with diversified instance types across multiple Availability Zones.
  • Set a maximum bid price at or below the on-demand price.
  • Use persistent requests so instances restart when capacity returns.
  • Combine Spot with On-Demand instances for critical workloads (minimum 20% On-Demand).

Common Mistakes with ec2 spot termination

  1. Non-exhaustive pattern matches that compile with warnings then crash at runtime
  2. Misunderstanding that String is [Char] with poor performance for large text operations
  3. Using foldl instead of foldl' causing stack overflow on large lists

These mistakes appear frequently in real-world AWS code. DodaTech's contributors have identified these patterns through analysis of open-source projects and production systems.

Practice Exercise

Write a pure function that safely divides two integers using Maybe, then test it with edge cases like division by zero and negative numbers.

This exercise reinforces the concepts covered in this guide. Try implementing it before checking online solutions.

FAQ

How long before termination does AWS notify me?

AWS sends a termination notice 2 minutes before the actual termination via the instance metadata endpoint at http://169.254.169.254/latest/meta-data/spot/termination-time. |||Can I stop a Spot Instance instead of terminating? Spot Instances can be stopped if you use a persistent request. The instance will stop instead of terminate when interrupted, then resume when capacity is available. |||What is the Spot price vs On-Demand savings? Spot Instances can save 60-90% compared to On-Demand pricing. The actual price fluctuates based on supply and demand, but you set a maximum bid to control costs.

← Previous Fix EC2 Spot Instance Interruption Configuration Next → How to Fix AWS EC2 SSH Connection Refused Error

Built by the developers of DodaTech

Doda Browser, DodaZIP & Durga Antivirus Pro

Home Browse Quick Fix