Celery Systemd: Running Celery Workers and Beat as Systemd Services

In this tutorial, you will learn about Celery Systemd: Running Celery Workers and Beat as Systemd Services. We cover key concepts, practical examples, and best practices to help you master this topic.

Celery systemd integration manages workers and Beat as native system services with automatic startup on boot, failure restart, resource limits via cgroups, centralized logging through journald, and environment-based configuration through service drop-ins.

flowchart LR
    Systemd[Systemd] --> W1[celery-worker.service]
    Systemd --> W2[celery-worker-default.service]
    Systemd --> Beat[celery-beat.service]
    Systemd --> Timer[celery-healthcheck.timer]
    W1 -->|journald| Logs[System Journal]
    W2 -->|journald| Logs
    Beat -->|journald| Logs
    W1 -->|cgroups| Resource[CPU/Memory Limits]

What You'll Learn

• Systemd service units for workers and Beat
• Environment file management
• Journald logging and log filtering
• Resource limits with cgroups
• Timer-based health checks

Why It Matters

Systemd is the standard init system for modern Linux distributions. Running Celery workers as systemd services provides boot-time startup, automatic restart, resource isolation, and integration with the system's logging and monitoring infrastructure.

Real-World Use

DodaTech deploys Celery workers as systemd services on bare-metal worker nodes. Each node runs 4 worker services with different queue assignments and resource limits. Journald collects all logs centrally, and systemd's restart logic handles crash recovery without additional tooling.

Worker Service Unit

# /etc/systemd/system/celery-worker.service
[Unit]
Description=Celery Worker - High Priority Queue
After=network.target redis.service
Wants=redis.service

[Service]
Type=forking
User=celeryuser
Group=celeryuser
WorkingDirectory=/opt/app
EnvironmentFile=-/etc/default/celery-worker
ExecStart=/opt/app/venv/bin/celery multi start worker-high \
    -A tasks \
    -Q high \
    --concurrency=8 \
    --loglevel=info \
    --logfile=/var/log/celery/worker-high.log \
    --pidfile=/var/run/celery/worker-high.pid
ExecStop=/opt/app/venv/bin/celery multi stop worker-high \
    --pidfile=/var/run/celery/worker-high.pid
ExecReload=/opt/app/venv/bin/celery multi restart worker-high \
    --pidfile=/var/run/celery/worker-high.pid
PIDFile=/var/run/celery/worker-high.pid
Restart=on-failure
RestartSec=10
StartLimitIntervalSec=60
StartLimitBurst=3

; Resource limits
LimitNOFILE=65536
LimitNPROC=4096
CPUQuota=50%
MemoryMax=2G
TasksMax=512

[Install]
WantedBy=multi-user.target

Environment file:

# /etc/default/celery-worker
CELERY_BROKER_URL=redis://localhost:6379/0
CELERY_RESULT_BACKEND=redis://localhost:6379/0
CELERYD_MAX_TASKS_PER_CHILD=1000

Enable and start:

sudo systemctl daemon-reload
sudo systemctl enable celery-worker
sudo systemctl start celery-worker
sudo systemctl status celery-worker

Expected output:

celery-worker.service - Celery Worker - High Priority Queue
   Loaded: loaded (/etc/systemd/system/celery-worker.service; enabled)
   Active: active (running) since Mon 2026-06-28 10:00:00 UTC
 Main PID: 12345 (celery)
   CGroup: /system.slice/celery-worker.service
           ├─12345 /opt/app/venv/bin/celery multi start ...
           ├─12346 celery worker ...

Beat Service Unit

# /etc/systemd/system/celery-beat.service
[Unit]
Description=Celery Beat Scheduler
After=network.target redis.service
Wants=redis.service

[Service]
Type=simple
User=celeryuser
Group=celeryuser
WorkingDirectory=/opt/app
EnvironmentFile=-/etc/default/celery-beat
ExecStart=/opt/app/venv/bin/celery -A tasks beat \
    --loglevel=info \
    --schedule=/var/run/celery/beat-schedule \
    --pidfile=/var/run/celery/beat.pid
ExecReload=/bin/kill -HUP $MAINPID
PIDFile=/var/run/celery/beat.pid
Restart=always
RestartSec=5

; Beat is lightweight, lower limits
LimitNOFILE=1024
MemoryMax=512M

[Install]
WantedBy=multi-user.target

Journald Logging

# /etc/systemd/journald.conf.d/celery.conf
[Journal]
MaxRetentionSec=30day
MaxFileSec=1day
SystemMaxUse=5G

# View Celery logs
# journalctl -u celery-worker.service -f
# journalctl -u celery-worker.service --since "1 hour ago"
# journalctl -u celery-worker.service -p err

journalctl -u celery-worker.service -f --output=json

Expected output:

{"PRIORITY":"6","SYSLOG_IDENTIFIER":"celery","MESSAGE":"Task process_file succeeded"}

Common Mistakes

• Type=forking vs Type=simple confusion -- Celery multi start forks workers into background. Use Type=forking with PIDFile for multi mode. Use Type=simple for single worker mode. Wrong type causes systemd to lose track of the worker PID.
• Not configuring After=redis.service -- workers start before Redis is ready and crash-loop. systemd restarts them (Restart=on-failure), but each restart attempt fails until Redis is available. Add After= and Wants= for the broker service.
• Missing LimitNOFILE -- Celery workers open many file descriptors (broker connections, result backend connections, log files). Default limit of 1024 causes "Too many open files" errors. Set LimitNOFILE=65536.
• Overlooking StartLimitIntervalSec -- without StartLimitIntervalSec and StartLimitBurst, a crash-looping worker can restart indefinitely, consuming resources. Set burst limits to detect persistent failures.
• Not using EnvironmentFile -- embedding environment variables (broker URLs, passwords) in service files makes them visible in systemctl status. Use EnvironmentFile pointing to a restricted-permissions file.

Practice Questions

1. What is the difference between Type=simple and Type=forking for Celery workers?
2. Why should you use EnvironmentFile instead of inline Environment directives?
3. How do you set memory limits for Celery workers in systemd?
4. How do you view Celery worker logs with journalctl?
5. What is the purpose of StartLimitIntervalSec and StartLimitBurst?

Challenge

Build a systemd-based Celery deployment: (1) service templates for worker (with configurable queue, concurrency, and log level), (2) Beat service with persistent schedule directory, (3) a celery.target that groups all Celery services for bulk start/stop, (4) timer-based health check unit that pings workers and restarts unresponsive ones, (5) log Rate Limiting via systemd journald config, (6) drop-in configuration files that override defaults per environment (staging, production).

FAQ

Mini Project

Build a systemd-based Celery management solution: (1) service template generator that creates .service and .environment files from a YAML configuration, (2) a celery.target that groups all Celery services with Wants/WantedBy for atomic start/stop, (3) a weekly log rotation config for journald that archives Celery-specific logs, (4) a healthcheck service and timer that runs celery inspect ping and restarts unresponsive workers, (5) Prometheus node_exporter integration that reports systemd service status, and (6) a deployment script that validates service files, reloads systemd, and performs rolling restarts.

What's Next

Continue with Monitoring with Prometheus to learn metrics-driven worker monitoring. Then explore Alerting and Troubleshooting for production issue response.