Cron Debugging — Complete Troubleshooting Guide for Cron Job Failures

In this tutorial, you will learn about Cron Debugging. We cover key concepts, practical examples, and best practices to help you master this topic.

Learn systematic cron debugging: check the cron daemon is running, inspect MAILTO output, verify crontab syntax with linting tools, test commands with minimal environment, and troubleshoot the most common silent failures.

What You Learn

You will learn step-by-step cron debugging techniques including daemon status checks, mail log inspection, crontab syntax validation, environment reproduction testing, and silent failure investigation.

Why It Matters

Cron job failures are often silent: the job runs but produces no output, or fails due to environment differences that only appear in cron's minimal shell. Knowing how to debug cron saves hours of guessing and prevents missed backups, alerts, and maintenance tasks.

Real-World Use

DodaTech's SRE team uses a standard cron debugging checklist for every incident. The most common fix is adding full PATH and environment variable exports to crontab. The second most common is fixing scripts that assume an interactive terminal.

Debugging Checklist

# 1. Is cron daemon running?
systemctl status cron
# or
service cron status
# or
ps aux | grep cron

# 2. Check crontab for current user
crontab -l

# 3. Check system crontabs
ls -la /etc/cron.d/
ls -la /etc/cron.hourly/
ls -la /etc/cron.daily/
ls -la /etc/cron.weekly/
ls -la /etc/cron.monthly/

# 4. Check cron mail (if MAILTO is set)
mail
# or check /var/mail/$USER

# 5. Check syslog for cron entries
grep CRON /var/log/syslog | tail -20
# or
journalctl -u cron --since "5 minutes ago"

# 6. Test the command manually
# Reproduce cron's minimal environment
env -i HOME=$HOME PATH=/usr/bin:/bin /bin/sh -c 'your-command'

Environment Reproduction

import os
import subprocess
import pprint

def debug_cron_environment(script_path):
    cron_env = {
        'HOME': os.environ.get('HOME', '/root'),
        'LOGNAME': os.environ.get('LOGNAME', 'root'),
        'PATH': '/usr/bin:/bin',
        'SHELL': '/bin/sh',
        'PWD': os.environ.get('HOME', '/root'),
    }

    print("Cron-like environment:")
    for key, value in sorted(cron_env.items()):
        print(f"  {key}={value}")

    print(f"\nRunning: {script_path}")
    result = subprocess.run(
        ['/bin/sh', '-c', script_path],
        env=cron_env,
        capture_output=True,
        text=True
    )

    print(f"Exit code: {result.returncode}")
    if result.stdout:
        print(f"STDOUT:\n{result.stdout}")
    if result.stderr:
        print(f"STDERR:\n{result.stderr}")

    return result.returncode == 0

debug_cron_environment('echo "Hello from cron environment" && python3 --version 2>&1 || echo "Python not in PATH"')

Expected output:

Cron-like environment:
  HOME=/root
  LOGNAME=root
  PATH=/usr/bin:/bin
  PWD=/root
  SHELL=/bin/sh

Running: echo "Hello from cron environment" && python3 --version 2>&1 || echo "Python not in PATH"
Exit code: 0
STDOUT:
Hello from cron environment
Python not in PATH

Silent Failure Detection

import subprocess
import time

class CronHealthCheck:
    def __init__(self, job_name, expected_interval_minutes):
        self.job_name = job_name
        self.expected_interval = expected_interval_minutes
        self.last_run_file = f"/tmp/cron_heartbeat_{job_name}"

    def mark_run(self):
        with open(self.last_run_file, 'w') as f:
            f.write(str(time.time()))
        print(f"[{self.job_name}] Heartbeat recorded")

    def check_missed(self):
        try:
            with open(self.last_run_file, 'r') as f:
                last_run = float(f.read().strip())
        except FileNotFoundError:
            print(f"[{self.job_name}] MISSED: No heartbeat file found")
            return True

        elapsed_minutes = (time.time() - last_run) / 60
        if elapsed_minutes > self.expected_interval * 2:
            print(f"[{self.job_name}] MISSED: Last run {elapsed_minutes:.0f} minutes ago")
            return True

        print(f"[{self.job_name}] OK: Last run {elapsed_minutes:.0f} minutes ago")
        return False

check = CronHealthCheck("db-backup", expected_interval_minutes=60)
check.mark_run()
time.sleep(0.1)
check.check_missed()

Expected output:

[db-backup] Heartbeat recorded
[db-backup] OK: Last run 0 minutes ago

Common Mistakes

1. Assuming Interactive Environment

Cron runs with a minimal environment: PATH=/usr/bin:/bin, no terminal, no X11. Scripts that work in your shell fail in cron because commands like python3 or docker are not in PATH. Always use full paths in cron scripts.

2. No Output Capture

If a cron job produces no output and MAILTO is not set, you never know if it ran or failed. Always redirect output: * * * * * /script.sh >> /var/log/cron/script.log 2>&1

3. Ignoring Exit Codes

A script may print an error message but exit with code 0. Cron only checks the exit code. Always set -e in shell scripts to fail on errors, or explicitly check exit codes.

4. Percent Signs Not Escaped

In crontab files, % has special meaning (newline in command). If your command includes % (like date +%Y), escape it as \% or put it in a separate script.

5. Symlinks in Cron Directories

/etc/cron.daily and similar directories only execute regular files, not symlinks. If you symlink a script into cron.daily, it will never run. Copy the file instead.

Practice Questions

1. What is the first thing to check when a cron job is not running?

Check the cron daemon status with systemctl status cron. If the daemon is not running, no Cron Jobs execute regardless of configuration.

2. Why do scripts work in terminal but fail in cron?

Cron provides a minimal environment: PATH=/usr/bin:/bin, no interactive shell rc files, no SSH agent, no terminal. Scripts must explicitly set PATH and other required variables.

3. How do you capture cron job output?

Redirect stdout and stderr to a log file: * * * * * /path/to/script.sh >> /var/log/myjob.log 2>&1. Alternatively, set MAILTO=email@example.com in crontab to receive output via email.

4. What does % mean in crontab?

In crontab, % represents a newline in the command. Everything after the first % is sent to the command's stdin. Escape as \% if you need a literal percent sign.

Challenge

Write a cron debugging script that: (1) prints the current cron daemon status and version, (2) lists all crontabs for all users (requires root), (3) shows the last 20 cron executions from syslog, (4) tests each crontab command with cron's minimal environment and reports which would fail, (5) checks for common issues (missing PATH, unescaped %, non-executable scripts, missing shebangs), (6) generates a JSON report of findings.

FAQ

Mini Project: Cron Debugging Toolkit

Build a cron debugging toolkit: (1) health check script that verifies cron daemon is running and responds, (2) crontab linter that validates syntax, checks for common issues (PATH, percent signs, full paths), (3) environment reproduction tool that runs commands in cron's minimal environment, (4) log watcher that tails syslog for cron entries and highlights failures, (5) heartbeat monitor that checks last run times against expected intervals, (6) report generator that outputs JSON summary of all cron jobs and their health status.

What's Next

Now that you understand cron debugging, explore testing cron jobs systematically, then learn about cron job cleanup and retention policies.