Cron Backup Strategies — Automated Database and File System Backups

In this tutorial, you will learn about Cron Backup Strategies. We cover key concepts, practical examples, and best practices to help you master this topic.

Learn cron-based backup strategies: automate database backups with mysqldump and pg_dump, schedule file system backups with rsync, implement rotation and retention policies, replicate backups offsite, and verify backup integrity.

What You Learn

You will learn how to schedule and manage backups with cron: database dump scripts, file system snapshots, backup rotation with retention policies, offsite replication, and automated integrity verification.

Why It Matters

Backups are useless if they are not tested, not recent, or not restorable. Cron automation ensures backups run consistently, retention policies prevent disk exhaustion, and verification scripts confirm backups are restorable.

Real-World Use

DodaTech runs 10+ backup cron jobs per database: full backup every Sunday at 3 AM, incremental backups daily at midnight, Transaction log backups every 15 minutes. All backups are replicated to S3 with 30-day retention and verified with restore tests every week.

Database Backup Script

#!/bin/bash
# db-backup.sh — PostgreSQL backup with rotation
DB_NAME="${1:-dodatech}"
BACKUP_DIR="/var/backups/postgresql"
RETENTION_DAYS=30
TIMESTAMP=$(date +%Y%m%d_%H%M%S)
BACKUP_FILE="${BACKUP_DIR}/${DB_NAME}_${TIMESTAMP}.sql.gz"

mkdir -p "$BACKUP_DIR"

pg_dump "$DB_NAME" | gzip > "$BACKUP_FILE"
EXIT_CODE=$?

if [ $EXIT_CODE -eq 0 ]; then
    echo "[$(date)] Backup successful: $BACKUP_FILE ($(du -h "$BACKUP_FILE" | cut -f1))"
else
    echo "[$(date)] Backup FAILED for $DB_NAME (exit code: $EXIT_CODE)"
    exit 1
fi

# Rotate old backups
find "$BACKUP_DIR" -name "${DB_NAME}_*.sql.gz" -mtime +$RETENTION_DAYS -delete
echo "[$(date)] Cleaned backups older than ${RETENTION_DAYS} days"

import os
import time
from datetime import datetime, timedelta
import hashlib
import json

class BackupManager:
    def __init__(self, backup_dir, retention_days=30):
        self.backup_dir = backup_dir
        self.retention_days = retention_days
        os.makedirs(backup_dir, exist_ok=True)

    def create_backup(self, name, data_generator):
        timestamp = datetime.now().strftime("%Y%m%d_%H%M%S")
        filename = f"{name}_{timestamp}.bak"
        filepath = os.path.join(self.backup_dir, filename)

        start = time.time()
        data = data_generator()
        with open(filepath, 'w') as f:
            f.write(json.dumps(data))
        duration = time.time() - start

        checksum = hashlib.md5(open(filepath, 'rb').read()).hexdigest()
        size = os.path.getsize(filepath)

        manifest = {
            "filename": filename,
            "size_bytes": size,
            "checksum_md5": checksum,
            "duration_seconds": round(duration, 2),
            "created_at": timestamp,
        }
        with open(filepath + ".manifest", 'w') as f:
            json.dump(manifest, f)

        print(f"Backup created: {filename} ({size / 1024:.1f} KB, {duration:.1f}s)")
        return manifest

    def rotate_old_backups(self):
        cutoff = time.time() - (self.retention_days * 86400)
        removed = 0
        for f in os.listdir(self.backup_dir):
            fpath = os.path.join(self.backup_dir, f)
            if os.path.isfile(fpath) and os.path.getmtime(fpath) < cutoff:
                os.remove(fpath)
                removed += 1
        print(f"Removed {removed} files older than {self.retention_days} days")
        return removed

manager = BackupManager("/tmp/test_backups", retention_days=30)
manager.create_backup("app_config", lambda: {"users": 100, "settings": {"theme": "dark"}})
manager.rotate_old_backups()

Expected output:

Backup created: app_config_20260628_000000.bak (0.1 KB, 0.0s)
Removed 0 files older than 30 days

Backup Verification

import hashlib
import json
import os

class BackupVerifier:
    def __init__(self):
        self.checks = []

    def verify_backup(self, backup_file, manifest_file):
        if not os.path.exists(backup_file):
            return {"file": backup_file, "status": "MISSING", "errors": ["File not found"]}

        if not os.path.exists(manifest_file):
            return {"file": backup_file, "status": "NO_MANIFEST", "errors": ["Manifest missing"]}

        with open(manifest_file) as f:
            manifest = json.load(f)

        errors = []
        actual_size = os.path.getsize(backup_file)
        if actual_size != manifest.get("size_bytes", 0):
            errors.append(f"Size mismatch: expected {manifest['size_bytes']}, got {actual_size}")

        actual_checksum = hashlib.md5(open(backup_file, 'rb').read()).hexdigest()
        if actual_checksum != manifest.get("checksum_md5", ""):
            errors.append(f"Checksum mismatch: expected {manifest['checksum_md5']}, got {actual_checksum}")

        status = "VERIFIED" if not errors else "CORRUPT"
        return {"file": backup_file, "status": status, "errors": errors}

verifier = BackupVerifier()
result = verifier.verify_backup("/tmp/test_backups/nonexistent.bak", "/tmp/test_backups/nonexistent.manifest")
print(json.dumps(result, indent=2))

Expected output:

{
  "file": "/tmp/test_backups/nonexistent.bak",
  "status": "MISSING",
  "errors": ["File not found"]
}

Common Mistakes

1. No Backup Verification

A backup that silently fails produces a zero-byte file that looks like a backup but restores nothing. Always verify backup files: check file size > 0, validate checksums, and periodically test a full restore.

2. Retention Without Rotation

Without rotation, backups accumulate until the disk fills. Implement retention policies: daily backups keep 7 days, weekly backups keep 4 weeks, monthly backups keep 12 months. Use cron to delete expired backups.

3. Backups on Same Disk as Data

A backup stored on the same disk as the source data is useless if the disk fails. Always store backups on a different volume, server, or cloud storage. Implement 3-2-1 backup rule: 3 copies, 2 media types, 1 offsite.

4. No Monitoring of Backup Size

If a database grows but the backup file stays the same size, the backup may be failing silently. Monitor backup file sizes over time and alert on unexpected changes.

5. Restore Never Tested

The only way to know a backup works is to restore it. Schedule monthly restore tests: restore the most recent backup to a staging environment, run integrity checks, and verify data completeness.

Practice Questions

1. What is the 3-2-1 backup rule?

3 copies of data, 2 different storage media, 1 copy offsite. For cron backups: primary copy on backup server, secondary copy on different volume, tertiary copy in cloud storage.

2. How do you verify a database backup?

Check file size is non-zero, validate gzip integrity (gzip -t), restore to a test database and run checksum queries comparing row counts with production. Automated verification should run after every backup.

3. What is the difference between full and incremental backups?

Full backup copies all data. Incremental backup copies only data changed since the last full or incremental backup. Full backups take longer but restore faster. Incremental backups are faster but require all increments for restoration.

4. How do you implement backup rotation with cron?

Use find -mtime to delete backups older than the retention period. Run rotation immediately after creating new backups. Store retention periods in a config file: daily=7, weekly=30, monthly=365.

Challenge

Build a complete backup system: (1) full backup cron (weekly Sunday 3 AM), incremental backup cron (daily Monday-Saturday 3 AM), transaction log backup (every 15 minutes), (2) backup verification after each full backup (restore to test DB, compare row counts), (3) rotation: keep 4 full backups, 7 daily incrementals, 30 days of transaction logs, (4) offsite replication: rsync or s3 sync after each full backup, (5) monitoring: backup duration, size, success/failure, time since last successful backup, (6) alerting: notify on failure, size anomaly, or skipped schedule.

FAQ

Mini Project: Automated Backup System

Build a cron-based backup automation system: (1) backup scheduler: full backup weekly, incremental daily, transaction logs every 15 minutes, (2) backup manager: configurable backup types (database dump, file system tar, rsync mirror), (3) compression with gzip with parallel support, (4) verification: file size check, checksum validation, test restore for full backups, (5) rotation: retention policies per backup type with automatic cleanup, (6) offsite replication: sync to S3 with encryption, (7) monitoring: Prometheus metrics for backup status, duration, size, and age, (8) alerting: Slack notification on failure, PagerDuty on consecutive failures.

What's Next

Now that you understand cron backup strategies, explore database maintenance scheduling, then learn about log rotation with cron.