Cron Cleanup Strategies — Managing Temporary Files and Job Artifacts

In this tutorial, you will learn about Cron Cleanup Strategies. We cover key concepts, practical examples, and best practices to help you master this topic.

Learn cron cleanup strategies for managing temporary files, log rotation, database archival, disk space monitoring, and automated retention policies using cron-scheduled cleanup jobs.

What You Learn

You will learn how to schedule automated cleanup jobs with cron: temporary file deletion, log rotation and compression, database archival and purging, disk space threshold monitoring, and retention policy implementation.

Why It Matters

Without cleanup jobs, temporary files accumulate, logs fill disks, and databases grow unbounded. A single cron cleanup script running daily prevents disk-full incidents that cause application outages.

Real-World Use

DodaTech runs cleanup cron jobs every 6 hours: temporary build artifacts older than 1 hour are deleted, application logs older than 30 days are compressed, database audit logs older than 90 days are archived to S3, and a disk usage alert triggers when any partition exceeds 85% capacity.

Temp File Cleanup

#!/bin/bash
# cleanup-temp.sh — Remove temp files older than N hours
TEMP_DIRS=("/tmp" "/var/tmp" "/app/cache")
MAX_AGE_HOURS=24

for dir in "${TEMP_DIRS[@]}"; do
    if [ -d "$dir" ]; then
        COUNT=$(find "$dir" -type f -mmin +$((MAX_AGE_HOURS * 60)) | wc -l)
        find "$dir" -type f -mmin +$((MAX_AGE_HOURS * 60)) -delete 2>/dev/null
        echo "[$(date)] Cleaned $COUNT files from $dir (older than ${MAX_AGE_HOURS}h)"
    fi
done

import os
import time
from pathlib import Path

def cleanup_old_files(directory, max_age_hours=24, dry_run=False):
    now = time.time()
    max_age_seconds = max_age_hours * 3600
    cleaned = 0
    bytes_freed = 0

    for root, dirs, files in os.walk(directory):
        for name in files:
            filepath = os.path.join(root, name)
            try:
                age = now - os.path.getmtime(filepath)
                if age > max_age_seconds:
                    size = os.path.getsize(filepath)
                    if not dry_run:
                        os.remove(filepath)
                    cleaned += 1
                    bytes_freed += size
            except OSError:
                continue

    return cleaned, bytes_freed

cleaned, freed = cleanup_old_files("/tmp/test_cleanup", max_age_hours=0, dry_run=True)
print(f"Would clean: {cleaned} files ({freed / 1024:.1f} KB freed)")

Expected output:

Would clean: 0 files (0.0 KB freed)

Database Cleanup Cron

import time
from datetime import datetime, timedelta

class DatabaseCleanupCron:
    def __init__(self):
        self.tables = []

    def add_table(self, name, retention_days, archive_before_delete=False):
        self.tables.append({
            'name': name,
            'retention_days': retention_days,
            'archive': archive_before_delete,
        })

    def run_cleanup(self):
        print(f"Database cleanup started at {datetime.now().isoformat()}")
        for table in self.tables:
            cutoff = datetime.now() - timedelta(days=table['retention_days'])
            print(f"  Purging {table['name']}: records before {cutoff.date()}")
            if table['archive']:
                print(f"    Archiving to S3 before delete")

            rows_deleted = 100
            print(f"    Deleted {rows_deleted} rows from {table['name']}")
        print("Cleanup complete.")

cleaner = DatabaseCleanupCron()
cleaner.add_table("audit_logs", retention_days=90, archive_before_delete=True)
cleaner.add_table("sessions", retention_days=7)
cleaner.add_table("api_requests", retention_days=30)
cleaner.run_cleanup()

Expected output:

Database cleanup started at 2026-06-28T00:00:00
  Purging audit_logs: records before 2026-03-30
    Archiving to S3 before delete
    Deleted 100 rows from audit_logs
  Purging sessions: records before 2026-06-21
    Deleted 100 rows from sessions
  Purging api_requests: records before 2026-05-29
    Deleted 100 rows from api_requests

Common Mistakes

1. Deleting Files Based on Name Instead of Age

A cleanup that deletes *.tmp but not *.log misses log files that are also temporary. Clean up based on file age and retention policy, not file extension. Use find -mtime or find -mmin for age-based cleanup.

2. No Dry-Run Mode

A typo in a cleanup cron job can delete production data. Always implement a dry-run mode that reports what would be deleted without actually deleting. Run the dry-run, verify the output, then remove the flag.

3. Cleanup During Business Hours

Deleting temporary files while applications are actively writing them causes errors. Schedule cleanup jobs during low-traffic periods. For critical systems, implement a quiesce mode that pauses cleanup while files are actively in use.

4. Not Checking for Open File Handles

A cleanup job may delete a file that a running Process has open. The file appears deleted but space is not freed until the process closes it. Use lsof to check for open file handles before deleting, or use truncation instead of deletion for active logs.

5. No Cleanup Monitoring

If a cleanup job fails, files accumulate unnoticed. Monitor cleanup jobs: bytes freed per run, files deleted, errors encountered. Alert if cleanup frees 0 bytes for 3 consecutive runs (indicates either nothing to clean or the job is broken).

Practice Questions

1. How do you determine which files to clean up in a cron job?

Use file age (last modification time), not file extension or name. Set retention policies per directory: /tmp = 1 hour, /var/tmp = 24 hours, logs = 30 days, archives = 90 days.

2. What is the safest way to implement a cleanup cron job?

Always include a dry-run mode. Log what would be deleted. Run the dry-run first in production, verify the output, then remove the --dry-run flag. Schedule destructive operations during maintenance Windows.

3. How do you handle cleanup of database audit logs?

Archive before deleting: export records older than the retention period to S3 or cold storage, verify the archive, then delete from the database. Schedule during low-traffic periods to minimize impact.

4. Why should cleanup jobs be monitored?

Cleanup failures lead to disk-full incidents that cause application outages. Monitor bytes freed per run, files deleted, and error counts. Alert if free disk space drops below thresholds despite cleanup jobs.

Challenge

Design a comprehensive cleanup system: (1) temp file cleanup every hour (files older than 1 hour in /tmp, 24 hours in /var/tmp), (2) log rotation with compression daily (30-day retention, gzip rotated logs), (3) database archival weekly (archive audit logs older than 90 days to S3, purge after archive confirmation), (4) disk monitoring every 5 minutes (alert at 85%, critical at 95%), (5) dry-run mode for all operations, (6) structured metrics logging for each cleanup operation, (7) Prometheus metrics for bytes freed, files processed, and error counts.

FAQ

Mini Project: Automated Cleanup System

Build an automated cleanup system with cron: (1) temp file cleanup: find and delete files in /tmp and /var/tmp older than configurable thresholds, (2) log rotation: rotate, compress, and delete application logs based on retention policy, (3) database purge: archive old records to cold storage and delete from production, (4) disk monitoring: check disk usage every 5 minutes, alert at configured thresholds, (5) all cleanup operations support dry-run mode, (6) structured JSON logging per operation, (7) metrics exposed for Prometheus with bytes freed, files processed, errors, and duration.

What's Next

Now that you understand cron cleanup strategies, explore cron job notifications, then learn about scheduling backups with cron.