Logging Cron Jobs — Complete Guide
In this tutorial, you will learn about Logging Cron Jobs. We cover key concepts, practical examples, and best practices to help you master this topic.
Implement proper logging for cron jobs: redirect stdout and stderr, use syslog, structured logging, log rotation, centralized log aggregation, and monitoring cron output.
What You Learn
You will learn how to capture cron job output, redirect logs to files, use syslog for system integration, implement structured logging, rotate logs to prevent disk exhaustion, and aggregate logs from multiple servers.
Why It Matters
Cron jobs run silently. When they fail, you need logs to diagnose the problem. Without proper logging, failures go unnoticed, and debugging requires re-running and hoping the failure reproduces.
Real-World Use
DodaTech's cron logging pipeline captures output from 200+ cron jobs across 50 servers. Logs are shipped to a central Elasticsearch cluster, with alerts for any job that exits with a non-zero code or produces stderr output.
Basic Output Redirection
# Redirect stdout and stderr to a log file
# >> appends, > overwrites
0 3 * * * /usr/local/bin/backup.sh >> /var/log/cron/backup.log 2>&1
# Separate stdout and stderr to different files
0 3 * * * /usr/local/bin/backup.sh \
>> /var/log/cron/backup.log \
2>> /var/log/cron/backup-err.log
# Discard all output (not recommended)
0 3 * * * /usr/local/bin/backup.sh > /dev/null 2>&1
# Log with timestamp prefix
0 3 * * * echo "[$(date)]" >> /var/log/cron/backup.log \
&& /usr/local/bin/backup.sh >> /var/log/cron/backup.log 2>&1
Logging with logger Command
# Use logger to send output to syslog
0 3 * * * /usr/local/bin/backup.sh 2>&1 | logger -t backup-job
# Check syslog for cron output
grep "backup-job" /var/log/syslog
# logger with facility and priority
0 3 * * * /usr/local/bin/backup.sh 2>&1 | \
logger -t backup-job -p cron.info
# On failure
0 3 * * * /usr/local/bin/backup.sh || \
logger -t backup-job -p cron.err "Backup failed with exit code $?"
# Use logger from within scripts
#!/bin/bash
# log-example.sh
logger -t my-cron "Starting maintenance task"
# ... do work ...
if [ $? -eq 0 ]; then
logger -t my-cron "Maintenance completed successfully"
else
logger -t my-cron -p cron.err "Maintenance failed"
fi
Structured Logging from Cron
#!/usr/bin/env python3
"""Cron job with structured JSON logging."""
import json
import sys
import logging
import os
from datetime import datetime
class CronLogger:
def __init__(self, job_name, log_file=None):
self.job_name = job_name
self.log_file = log_file or f"/var/log/cron/{job_name}.json"
self.logger = logging.getLogger(job_name)
self._setup()
def _setup(self):
handler = logging.FileHandler(self.log_file)
handler.setFormatter(logging.Formatter(
'%(message)s'
))
self.logger.addHandler(handler)
self.logger.setLevel(logging.INFO)
def _log(self, level, message, extra=None):
entry = {
'timestamp': datetime.utcnow().isoformat(),
'job': self.job_name,
'level': level,
'message': message,
'pid': os.getpid(),
'host': os.uname().nodename,
}
if extra:
entry.update(extra)
if level == 'ERROR':
self.logger.error(json.dumps(entry))
else:
self.logger.info(json.dumps(entry))
def info(self, message, **extra):
self._log('INFO', message, extra)
def error(self, message, **extra):
self._log('ERROR', message, extra)
def main():
log = CronLogger('daily-backup')
log.info("Backup started", database="mydb", size_mb=1024)
try:
# Simulate backup work
result = {'exit_code': 0, 'files': 42, 'size_mb': 1024}
log.info("Backup completed", **result)
except Exception as e:
log.error("Backup failed", error=str(e))
sys.exit(1)
if __name__ == '__main__':
main()
Log Rotation
# Logrotate configuration for cron logs
# /etc/logrotate.d/cron-jobs
/var/log/cron/*.log {
daily
rotate 30
compress
delaycompress
missingok
notifempty
create 0644 root root
sharedscripts
postrotate
# Reload logger if needed
endscript
}
/var/log/cron/*.json {
daily
rotate 90
compress
delaycompress
missingok
notifempty
create 0644 root root
}
# Test logrotate configuration:
# sudo logrotate -d /etc/logrotate.d/cron-jobs
# Force rotation:
# sudo logrotate -f /etc/logrotate.d/cron-jobs
Centralized Log Aggregation
# Using syslog-ng or rsyslog to forward cron logs
# rsyslog configuration (/etc/rsyslog.d/cron-forward.conf)
# Forward cron logs to central server
cron.* @logserver.dodatech.com:514
cron.* @@logserver.dodatech.com:10514 # TCP with TLS
# Using filebeat to ship cron logs to Elasticsearch
# /etc/filebeat/filebeat.yml
# filebeat.inputs:
# - type: log
# enabled: true
# paths:
# - /var/log/cron/*.log
# - /var/log/cron/*.json
# multiline.pattern: '^\[' # Combine multi-line logs
# multiline.negate: true
# multiline.match: after
#
# output.elasticsearch:
# hosts: ["elasticsearch.dodatech.com:9200"]
# Using Vector for log aggregation:
# /etc/vector/vector.toml
# [sources.cron_logs]
# type = "file"
# include = ["/var/log/cron/*.log"]
#
# [sinks.elasticsearch]
# type = "elasticsearch"
# inputs = ["cron_logs"]
# endpoint = "https://elasticsearch.dodatech.com:9200"
Cron Log Monitoring
#!/usr/bin/env python3
"""Monitor cron logs for failures and send alerts."""
import os
import re
import time
import subprocess
from datetime import datetime, timedelta
class CronLogMonitor:
def __init__(self, log_dir='/var/log/cron', alert_script=None):
self.log_dir = log_dir
self.alert_script = alert_script or '/usr/local/bin/alert.sh'
self.position_file = '/tmp/cron-monitor-pos.txt'
self.positions = self._load_positions()
def _load_positions(self):
positions = {}
if os.path.exists(self.position_file):
with open(self.position_file) as f:
for line in f:
parts = line.strip().split(':')
if len(parts) == 2:
positions[parts[0]] = int(parts[1])
return positions
def _save_positions(self):
with open(self.position_file, 'w') as f:
for name, pos in self.positions.items():
f.write(f"{name}:{pos}\n")
def check_logs(self):
for filename in os.listdir(self.log_dir):
filepath = os.path.join(self.log_dir, filename)
if not os.path.isfile(filepath):
continue
position = self.positions.get(filename, 0)
current_size = os.path.getsize(filepath)
if current_size <= position:
continue
with open(filepath) as f:
f.seek(position)
for line in f:
if 'error' in line.lower() or 'failed' in line.lower() or 'exit code' in line.lower():
self._alert(filename, line.strip())
self.positions[filename] = current_size
self._save_positions()
def _alert(self, log_file, message):
timestamp = datetime.now().isoformat()
alert_msg = f"[{timestamp}] CRON ALERT: {log_file}: {message}"
print(alert_msg)
if os.path.exists(self.alert_script):
subprocess.run([self.alert_script, alert_msg])
monitor = CronLogMonitor()
monitor.check_logs()
Expected output:
[2026-06-28T03:15:00] CRON ALERT: backup.log: ERROR: Database connection failed
Common Mistakes
1. Not Capturing stderr
Many cron jobs only redirect stdout, missing error messages. Always use 2>&1 to capture stderr alongside stdout.
2. Overwriting Log Files
Using > instead of >> overwrites the log file each run. Use >> to append. Overwritten logs lose history.
3. No Log Rotation
Logs grow indefinitely and fill the disk. Configure logrotate or delete old logs in the cron script itself.
4. Logging to /dev/null
Discarding all output makes debugging impossible when jobs fail. Always log to a file or syslog.
5. No Timestamps
Without timestamps, correlating log entries with specific job runs is impossible. Prefix each log line with a timestamp.
Practice Questions
1. How do you redirect both stdout and stderr to a log file?
Use >> /path/to/logfile 2>&1 at the end of the cron command. The 2>&1 redirects stderr (file descriptor 2) to stdout (file descriptor 1).
2. What is the logger command?
logger sends messages to syslog. It is useful for cron jobs because output is automatically integrated with system logging and can be forwarded to central log servers.
3. Why is log rotation important for cron jobs?
Without rotation, log files grow indefinitely and fill the disk. logrotate manages log file size by compressing, archiving, and deleting old logs.
4. How do you aggregate cron logs from multiple servers?
Use syslog forwarding (rsyslog, syslog-ng), or a log shipper (Filebeat, Vector, Fluentd) that sends logs to a central Elasticsearch, Loki, or Splunk cluster.
Challenge
Build a cron logging system: each job writes structured JSON logs to /var/log/cron/jobname/date.json, logs are rotated daily with 90-day retention, errors trigger an alert via Webhook to Slack, and all logs are shipped to a central Elasticsearch cluster.
FAQ
{{< faq "How do I prevent cron log email spam?" "Set MAILTO="" in crontab to disable email. Or redirect output to a log file and cron will not send email because there is no output." >}}
Mini Project: Cron Logging Wrapper
#!/bin/bash
# /usr/local/bin/cron-log-wrapper.sh
# Usage: crontab entry:
# 0 3 * * * /usr/local/bin/cron-log-wrapper.sh /usr/local/bin/backup.sh
JOB_NAME=$(basename "$1")
LOG_DIR="/var/log/cron"
LOG_FILE="${LOG_DIR}/${JOB_NAME}.log"
TIMESTAMP=$(date '+%Y-%m-%d %H:%M:%S')
# Ensure log directory exists
mkdir -p "$LOG_DIR"
# Log start
echo "[${TIMESTAMP}] START: ${JOB_NAME} (PID: $$)" >> "$LOG_FILE"
# Execute job, capture exit code
START_TIME=$(date +%s)
"$@" >> "$LOG_FILE" 2>&1
EXIT_CODE=$?
END_TIME=$(date +%s)
DURATION=$((END_TIME - START_TIME))
# Log completion
if [ $EXIT_CODE -eq 0 ]; then
echo "[$(date '+%Y-%m-%d %H:%M:%S')] END: ${JOB_NAME} (exit: 0, duration: ${DURATION}s)" >> "$LOG_FILE"
else
echo "[$(date '+%Y-%m-%d %H:%M:%S')] FAIL: ${JOB_NAME} (exit: ${EXIT_CODE}, duration: ${DURATION}s)" >> "$LOG_FILE"
# Send alert on failure
logger -t cron-wrapper -p cron.err "${JOB_NAME} failed with exit code ${EXIT_CODE}"
fi
exit $EXIT_CODE
What's Next
Now that you understand cron logging, explore error handling in cron for managing failed jobs, then learn about locking and concurrency control for preventing overlapping executions.
Built by the developers of DodaTech
Doda Browser, DodaZIP & Durga Antivirus Pro