Celery Late Acknowledgement: Reliable Task Execution with Delivery Guarantees

In this tutorial, you will learn about Celery Late Acknowledgement: Reliable Task Execution with Delivery Guarantees. We cover key concepts, practical examples, and best practices to help you master this topic.

Celery late acknowledgement defers the broker message acknowledgement until after the task completes, guaranteeing that no task is lost if a worker crashes mid-execution by making the broker redeliver unacknowledged messages to other workers.

flowchart LR
    B[Broker] -->|Task Message| W1[Worker]
    W1 -->|Ack after completion| B
    W1 -->|Crash!| X{Worker Fails}
    X -->|No ack sent| B
    B -->|Redeliver| W2[Worker 2]
    W2 -->|Ack after completion| B
    style X fill:#ff4444,color:#fff

What You'll Learn

• Late ack configuration and behavior
• At-least-once vs at-most-once delivery
• Task redelivery after worker failure
• Interaction with time limits and retries
• Late ack performance considerations

Why It Matters

Without late ack, a worker crash during task execution causes permanent message loss. Late ack ensures tasks are delivered at least once, making it essential for payment processing, data migrations, and any operation that must not lose work items.

Real-World Use

DodaTech's malware analysis platform uses late ack for file scanning tasks. If a worker crashes mid-scan, the unprocessed file is redelivered to another worker within minutes. Without late ack, uploaded files would silently disappear from the processing pipeline.

Configuring Late Ack

from celery import Celery

app = Celery('lateack', broker='redis://localhost:6379/0')

app.conf.task_acks_late = True
app.conf.worker_prefetch_multiplier = 1
app.conf.broker_transport_options = {
    'visibility_timeout': 3600,
    'max_retries': 3,
}

@app.task(bind=True, max_retries=3)
def process_payment(payment_id):
    import time
    time.sleep(0.1)
    result = f"Payment {payment_id} processed"
    print(result)
    return result

task = process_payment.delay("PAY-001")
print(f"Task {task.id} submitted with late ack")

Expected output:

Task 550e8400-e29b-41d4-a716-446655440000 submitted with late ack
Payment PAY-001 processed
[2026-06-28 10:00:00: INFO] Task process_payment succeeded
[2026-06-28 10:00:00: INFO] ack: process_payment acknowledged after completion

Redelivery After Crash

from celery import Celery
import os
import time

app = Celery('lateack', broker='redis://localhost:6379/0')
app.conf.task_acks_late = True
app.conf.worker_prefetch_multiplier = 1

SIMULATE_CRASH = os.environ.get('SIMULATE_CRASH', '0') == '1'
CRASH_AFTER = int(os.environ.get('CRASH_AFTER', '3'))

task_count = {'value': 0}

@app.task(bind=True)
def fragile_task(self, item_id):
    task_count['value'] += 1
    count = task_count['value']
    print(f"Processing item {item_id} (attempt {count})")

    if SIMULATE_CRASH and count >= CRASH_AFTER:
        print(f"Simulating crash after {count} tasks...")
        os._exit(1)

    time.sleep(0.2)
    result = f"Item {item_id} completed"
    print(result)
    return result

for i in range(5):
    fragile_task.delay(i)
print("Submitted 5 tasks with crash simulation")

Expected output when run with SIMULATE_CRASH=1:

Submitted 5 tasks with crash simulation
Processing item 0 (attempt 1)
Item 0 completed
Processing item 1 (attempt 2)
Item 1 completed
Processing item 2 (attempt 3)
Simulating crash after 3 tasks...
[Worker restarts]
Processing item 2 (attempt 4)
Item 2 completed
Processing item 3 (attempt 5)
Processing item 4 (attempt 6)
Item 3 completed
Item 4 completed

Prefetch and Late Ack Interaction

from celery import Celery

app = Celery('lateack', broker='redis://localhost:6379/0')
app.conf.task_acks_late = True

app.conf.worker_prefetch_multiplier = 1

@app.task(bind=True)
def prefetch_demo(task_id):
    import time
    time.sleep(0.5)
    result = f"Task {task_id} completed"
    print(result)
    return result

for i in range(4):
    prefetch_demo.delay(i + 1)
print("Submitted 4 tasks with prefetch_multiplier=1")

Expected output:

Submitted 4 tasks with prefetch_multiplier=1
Task 1 completed
Task 2 completed
Task 3 completed
Task 4 completed

Each task is fetched only after the previous one completes and is acknowledged.

Common Mistakes

• High prefetch multiplier with late ack -- setting worker_prefetch_multiplier above 1 with late ack causes the worker to prefetch many messages. If the worker crashes, all prefetched but unacknowledged messages must wait for the visibility timeout before redelivery.
• Late ack with task time limits -- a task that hits its time limit and raises SoftTimeLimitExceeded is acknowledged on error. The broker considers the task processed and does not redeliver. Handle timeouts within the task if redelivery is needed.
• Not setting visibility timeout -- Redis and SQS brokers have a default visibility timeout (Redis: 1 hour, SQS: 30 seconds). If your tasks take longer, set visibility_timeout higher to prevent premature redelivery.
• Late ack without idempotent tasks -- if a task crashes after side effects (like sending an email) but before acknowledgement, it will be redelivered and run again. Ensure tasks are idempotent to handle duplicate execution.
• Enabling late ack on non-critical tasks -- late ack adds overhead and complexity. Only enable it for tasks where message loss is unacceptable. Use default ack for tasks that can safely lose a message.

Practice Questions

1. What is the difference between early ack and late ack in Celery?
2. How does worker_prefetch_multiplier affect late ack behavior?
3. What happens to prefetched tasks when a worker crashes with late ack enabled?
4. Why must tasks be idempotent when using late ack?
5. How do you set the visibility timeout for Redis broker with late ack?

Challenge

Build a task processing system with late ack that: (1) processes 100 payment transactions, (2) simulates random worker crashes every 10-15 tasks, (3) ensures no Transaction is processed more than once using idempotency keys stored in Redis, (4) reports which tasks were redelivered and how many times, and (5) cleans up the idempotency keys after 24 hours.

FAQ

Mini Project

Build a reliable payment processing system using late ack: (1) simulate 1000 payment tasks submitted to Redis broker, (2) run 3 workers that Process tasks with late ack, (3) inject random worker crashes every 50 tasks, (4) track all redeliveries in a database, (5) verify zero message loss by comparing submitted vs completed counts, and (6) generate a redelivery report showing how many tasks required retry and average redelivery time.

What's Next

Continue with Task Expiration to learn about time-bound task execution. Then explore Soft and Hard Time Limits for task timeout management.