Nagios Service Check Hung — Quick Fix

In this tutorial, you'll learn about Nagios Service Check Hung. We cover key concepts, practical examples, and best practices.

The Problem

A Nagios service check is stuck in a "Pending" or "OK" state and never completes or updates. The check process may be hung indefinitely, consuming system resources. Nagios may eventually stop running new checks due to worker exhaustion.

Error example:

Warning: The check of service 'HTTP' on host 'web-server-01' looks like it might be hung.
Check process ID 12345 is not responding.
Service check timed out after 60 seconds.

The Fix

Step 1: Kill stuck check processes

WRONG — restarting Nagios without killing hung checks:

# Hung processes survive a Nagios restart and continue consuming resources

RIGHT — identify and kill hung check processes:

# Find check processes running longer than expected
ps aux | grep nagios | grep -v grep | awk '$8 ~ "R|D" {print $2, $11, $9}'

Output:

12345 check_http 15:23 (running for 15+ minutes)

Kill the hung process:

kill -9 12345

Step 2: Configure check timeouts in Nagios

WRONG — leaving the default timeout of 60 seconds for all checks:

# A network timeout can cause a check to hang indefinitely

RIGHT — set appropriate timeouts in nagios.cfg:

grep -E "^service_check_timeout|^host_check_timeout|^event_handler_timeout" /etc/nagios/nagios.cfg

Output:

service_check_timeout=60
host_check_timeout=30
event_handler_timeout=30

Reduce service_check_timeout to 30 seconds for faster hung-check detection.

Use DodaTech's Check Performance Monitor to identify slow checks, hung processes, and timeout-related alerts across your Nagios environment.

Prevention Tips

• Set service_check_timeout to 15-30 seconds max
• Configure child_processes_frequency to periodically reap stuck processes
• Monitor Nagios process count and alert if it exceeds expected values
• Use aggressive host checking to fail-fast when hosts go down
• Use DodaTech's Nagios Process Manager to automate hung check cleanup

Common Mistakes with service check hung

1. Overlapping type class instances that cause GHC to reject the program with ambiguous dispatch errors
2. Non-exhaustive pattern matches that compile with warnings then crash at runtime
3. Misunderstanding that String is [Char] with poor performance for large text operations

These mistakes appear frequently in real-world NAGIOS code. DodaTech's contributors have identified these patterns through analysis of open-source projects and production systems.

Practice Exercise

Write a pure function that safely divides two integers using Maybe, then test it with edge cases like division by zero and negative numbers.

This exercise reinforces the concepts covered in this guide. Try implementing it before checking online solutions.

FAQ

Related: DodaTech's Nagios Performance Tuning Guide covers check execution optimization, parallelization strategies, and timeout configuration best practices.