Celery Installation and Setup — Complete Guide

In this tutorial, you will learn about Celery Installation and Setup. We cover key concepts, practical examples, and best practices to help you master this topic.

Install Celery with pip, configure Redis as the broker, set up your first Celery app, and verify the worker processes tasks correctly.

What You Learn

You will install Celery and its dependencies, set up a Redis broker, create a Celery application, configure basic settings, and run your first worker.

Why It Matters

Correct installation and configuration are the foundation of any Celery system. A misconfigured broker or worker leads to tasks that are never executed, lost results, or mysterious failures that are difficult to debug.

Real-World Use

DodaTech's Celery deployment uses containerized workers with environment-specific configuration. The same codebase runs in development with Redis on localhost and in production with a managed Redis cluster.

Installing Celery

# Install Celery with Redis broker support
pip install celery[redis]

# Verify installation
python -c "import celery; print(celery.__version__)"

Expected output:

5.4.0

Setting Up Redis Broker

# Install Redis (Ubuntu/Debian)
sudo apt-get update
sudo apt-get install redis-server -y

# Start Redis
sudo systemctl start redis-server
sudo systemctl enable redis-server

# Verify Redis is running
redis-cli ping

Expected output:

PONG

Creating Your First Celery App

# celery_app.py
from celery import Celery

app = Celery(
    'myapp',
    broker='redis://localhost:6379/0',
    backend='redis://localhost:6379/0'
)

app.conf.update(
    task_serializer='json',
    accept_content=['json'],
    result_serializer='json',
    timezone='UTC',
    enable_utc=True,
    task_track_started=True,
    task_time_limit=30 * 60,
    task_soft_time_limit=25 * 60,
)

@app.task
def hello(name):
    return f"Hello, {name}!"

if __name__ == '__main__':
    result = hello.delay('DodaTech')
    print(f"Task ID: {result.id}")
    print(f"Result: {result.get(timeout=5)}")

Expected output:

Task ID: 550e8400-e29b-41d4-a716-446655440000
Result: Hello, DodaTech!

Running the Worker

Open a terminal and start the Celery worker:

celery -A celery_app worker --loglevel=info

Expected output:

-------------- celery@hostname v5.4.0 (dawn-chorus)
--- ***** -----
-- ******* ---- Linux-6.2.0-x86_64-with-glibc2.35 2026-06-28 10:00:00
- *** --- * ---
- ** ---------- [config]
- ** ---------- .> app:         myapp:0x7f...
- ** ---------- .> transport:   redis://localhost:6379/0
- ** ---------- .> results:     redis://localhost:6379/0
- *** --- * --- .> concurrency: 12 (prefork)
-- ******* ---- .> task events: OFF (enable -E to monitor tasks)
--- ***** -----
[2026-06-28 10:00:00: INFO/MainProcess] Connected to redis://localhost:6379/0
[2026-06-28 10:00:00: INFO/MainProcess] mingle: searching for neighbors
[2026-06-28 10:00:00: INFO/MainProcess] mingle: all alone
[2026-06-28 10:00:00: INFO/MainProcess] celery@hostname ready.

Configuration File

For larger projects, use a separate configuration file:

# celery_config.py
broker_url = 'redis://localhost:6379/0'
result_backend = 'redis://localhost:6379/0'
task_serializer = 'json'
result_serializer = 'json'
accept_content = ['json']
timezone = 'UTC'
enable_utc = True
task_track_started = True
task_time_limit = 1800
task_soft_time_limit = 1500
worker_max_tasks_per_child = 1000
worker_prefetch_multiplier = 1

# celery_app.py
from celery import Celery

app = Celery('myapp')
app.config_from_object('celery_config')

@app.task
def multiply(x, y):
    return x * y

Using Docker

# Dockerfile
FROM python:3.11-slim

WORKDIR /app

COPY requirements.txt .
RUN pip install -r requirements.txt

COPY . .

CMD ["celery", "-A", "celery_app", "worker", "--loglevel=info", "--concurrency=4"]

# docker-compose.yml
version: '3.8'

services:
  redis:
    image: redis:7-alpine
    ports:
      - "6379:6379"

  worker:
    build: .
    depends_on:
      - redis
    environment:
      - CELERY_BROKER_URL=redis://redis:6379/0
      - CELERY_RESULT_BACKEND=redis://redis:6379/0

Common Mistakes

1. Forgetting to Install Redis

Installing celery[redis] installs the Python client, but Redis server must be running separately. Workers fail to connect without a running broker.

2. Using Relative Imports in Tasks

Celery workers may change the working directory. Always use absolute imports or ensure Python path includes your project root.

3. Not Setting timezone

Celery defaults to UTC. If your application uses a different timezone, set it explicitly. This affects periodic task scheduling.

4. Running Workers Without --loglevel=info on First Run

Without loglevel info, you cannot see errors. Always start with --loglevel=info when debugging.

5. Forgetting to Restart Workers After Code Changes

Celery does not auto-reload on code changes by default. Use --autoreload in development or restart workers explicitly.

Practice Questions

1. How do you install Celery with Redis broker support?

Run pip install celery[redis]. This installs Celery, the Redis Python client, and all required dependencies.

2. What is the purpose of the Celery configuration file?

It centralizes settings like broker URL, result backend, serializers, time limits, and worker settings. Use app.config_from_object() to load it.

3. How do you start a Celery worker?

Run celery -A module_name worker --loglevel=info. The -A flag points to the Celery app instance.

4. What does the -E flag do when running a worker?

It enables task event monitoring. Events are required for tools like Flower to track task progress and worker status.

Challenge

Create a Docker Compose setup for Celery with three services: Redis, a Celery worker (4 processes), and a Celery Beat scheduler. Configure environment variables for broker URL and result backend.

FAQ

Mini Project: Dockerized Celery Setup

# app.py
from celery import Celery
import os

broker = os.environ.get('CELERY_BROKER_URL', 'redis://localhost:6379/0')
backend = os.environ.get('CELERY_RESULT_BACKEND', 'redis://localhost:6379/0')

app = Celery('docker_app', broker=broker, backend=backend)
app.conf.update(
    task_serializer='json',
    accept_content=['json'],
    result_serializer='json',
    timezone='UTC',
)

@app.task
def square(n):
    return n * n

# client.py
from app import square
import time

results = [square.delay(i) for i in range(10)]
for r in results:
    print(f"square({r.id[:8]}...) = {r.get(timeout=10)}")

Expected output:

square(550e8400...) = 0
square(6ba7b810...) = 1
square(6ba7b811...) = 4
square(6ba7b812...) = 9
square(6ba7b813...) = 16
square(6ba7b814...) = 25
square(6ba7b815...) = 36
square(6ba7b816...) = 49
square(6ba7b817...) = 64
square(6ba7b818...) = 81

What's Next

Now that Celery is installed and running, learn about broker setup with Redis and RabbitMQ for production configurations, then explore defining tasks with different options and configurations.