Airflow DAG Not Loading Fix

In this tutorial, you'll learn about Airflow DAG Not Loading Fix. We cover key concepts, practical examples, and best practices to help you understand and apply this topic effectively.

You place a DAG file in the dags folder but it doesn't appear in the Airflow UI:

DAGs page: "my_dag" is not listed.

Airflow's scheduler parses DAG files from the dags_folder. If the file has a syntax error, an import error, or doesn't create a DAG object with the correct naming convention, the scheduler silently skips it. The "Import Errors" section of the UI shows the exact failure.

Step-by-Step Fix

1. Check the Import Errors page

WRONG — refreshing the UI hoping it appears:

RIGHT — check the errors:

Airflow UI > DAGs > Import Errors (tab at the top)

This shows all DAG files that failed to parse with the exact error message. Common errors:

ModuleNotFoundError: No module named 'requests'
IndentationError: unexpected indent
NameError: name 'timedelta' is not defined

2. Test the DAG file locally

WRONG — relying on the scheduler to catch errors:

RIGHT — test from the command line:

# Test DAG parsing
python /path/to/dags/my_dag.py

# Or use Airflow's built-in checker
airflow dags test my_dag 2024-01-01

# Check DAG list
airflow dags list | grep my_dag

3. Verify the DAG object is named correctly

WRONG — Airflow requires the DAG object to be in globals():

from airflow import DAG
from datetime import datetime

my_dag = DAG(
    dag_id="my_dag",
    start_date=datetime(2024, 1, 1),
    schedule="@daily",
)  # This IS correct

RIGHT — ensure the DAG object is at module level:

dag = DAG(..., dag_id="my_dag")  # 'dag' is at module level

# These do NOT work:
def create_dag():
    dag = DAG(...)  # Local variable, not discovered

# Must be accessible as: from my_dag import dag

4. Fix missing dependencies

WRONG — the DAG imports packages not installed in the Airflow environment:

import requests  # Not installed on the Airflow server

RIGHT — install dependencies:

pip install requests

# Or add to requirements.txt
echo "requests==2.31.0" >> /path/to/requirements.txt
pip install -r /path/to/requirements.txt

Then restart the scheduler:

airflow scheduler restart

5. Check the DAG file location

WRONG — placing the DAG in a subfolder not scanned:

# Airflow scans dags_folder but NOT subdirectories by default

RIGHT — check the configuration:

airflow config get-value core dags_folder
# /path/to/dags

Place the DAG file directly in this folder, or enable subdirectory scanning:

airflow config set-value core dag_discovery_safe_mode False

6. Verify the DAG's start_date is in the past

WRONG — start_date in the future:

dag = DAG(
    dag_id="my_dag",
    start_date=datetime(2026, 12, 31),  # Future date!
    schedule="@daily",
)

RIGHT — use a past start_date:

dag = DAG(
    dag_id="my_dag",
    start_date=datetime(2024, 1, 1),
    schedule="@daily",
    catchup=False,
)

Expected output: the DAG appears in the Airflow UI.

Prevention

• Test DAG files locally with python dags/my_dag.py before deploying.
• Pin all Python dependencies in a requirements.txt.
• Use airflow dags list to verify the scheduler parsed your DAG.
• Set start_date in the past (yesterday or earlier).
• Check Import Errors in the UI after every DAG deployment.

Common Mistakes with dag not loading

1. Mixing let bindings with <- bindings in do notation, producing type errors
2. Overlapping type class instances that cause GHC to reject the program with ambiguous dispatch errors
3. Non-exhaustive pattern matches that compile with warnings then crash at runtime

These mistakes appear frequently in real-world AIRFLOW code. DodaTech's contributors have identified these patterns through analysis of open-source projects and production systems.

Practice Exercise

Write a pure function that safely divides two integers using Maybe, then test it with edge cases like division by zero and negative numbers.

This exercise reinforces the concepts covered in this guide. Try implementing it before checking online solutions.

FAQ