Skip to content

Python Logging — Complete Guide with Examples

DodaTech Updated 2026-06-29 5 min read

In this tutorial, you will learn about Python Logging. We cover key concepts, practical examples, and best practices to help you master this topic.

Print statements work for debugging on your laptop. In production, you need logging — the ability to record events with severity levels, timestamps, and structured context, routed to different destinations (console, files, external services).

Python's logging module is a production-ready logging framework built into the standard library. Every serious Python application uses it. At DodaTech, logging is critical for security auditing — every authentication attempt, file access, and configuration change is logged with structured detail.

What You'll Learn

  • Log levels: DEBUG, INFO, WARNING, ERROR, CRITICAL
  • Handlers: StreamHandler, FileHandler, RotatingFileHandler
  • Formatters: timestamps, loggers, structured output
  • Logger hierarchy and propagation
  • Configuration with dictConfig
  • Structured logging with JSON

Why Logging Matters

Without logging, when something breaks in production you have no idea what happened. With proper logging, you can trace the exact sequence of events leading to a failure. In security-sensitive applications like DodaTech's threat detection engine, every anomalous event is logged and correlated.

Basic Logging

import logging

# Simple configuration
logging.basicConfig(
    level=logging.INFO,
    format="%(asctime)s - %(name)s - %(levelname)s - %(message)s"
)

# Create a logger
logger = logging.getLogger(__name__)

# Log at different levels
logger.debug("This won't show — level is INFO")
logger.info("Application started")
logger.warning("Disk space low: 500MB remaining")
logger.error("Failed to connect to database: Connection refused")
logger.critical("System shutting down!")

Expected output:

2026-06-29 12:00:00,000 - __main__ - INFO - Application started
2026-06-29 12:00:00,001 - __main__ - WARNING - Disk space low: 500MB remaining
2026-06-29 12:00:00,002 - __main__ - ERROR - Failed to connect to database: Connection refused
2026-06-29 12:00:00,003 - __main__ - CRITICAL - System shutting down!

Log Levels Explained

Level Value When to Use
DEBUG 10 Detailed info for debugging (not in production)
INFO 20 Normal operation: "Server started", "User logged in"
WARNING 30 Something unexpected but not an error: "Disk at 80%"
ERROR 40 A failure that doesn't stop the app: "Failed to send email"
CRITICAL 50 The app can't continue: "Database connection lost"

Handlers and Formatters

Handlers send log records to different destinations:

import logging

logger = logging.getLogger("myapp")
logger.setLevel(logging.DEBUG)

# Console handler
console = logging.StreamHandler()
console.setLevel(logging.INFO)
console.setFormatter(logging.Formatter(
    "%(levelname)s: %(message)s"
))

# File handler with rotation
file_handler = logging.FileHandler("app.log")
file_handler.setLevel(logging.DEBUG)
file_handler.setFormatter(logging.Formatter(
    "%(asctime)s - %(name)s - %(levelname)s - %(message)s"
))

# Rotating file handler (keeps last 5 files, 1MB each)
from logging.handlers import RotatingFileHandler
rotating = RotatingFileHandler(
    "app.log", maxBytes=1024*1024, backupCount=5
)

logger.addHandler(console)
logger.addHandler(file_handler)

logger.info("This goes to both console and file")

Logger Hierarchy

Loggers are organized by name with dot-separated hierarchy:

import logging

# Root logger
root = logging.getLogger()
root.setLevel(logging.WARNING)

# Child loggers
app_logger = logging.getLogger("myapp")
db_logger = logging.getLogger("myapp.database")
api_logger = logging.getLogger("myapp.api")

# Messages propagate to parent loggers
api_logger.error("API request failed")  # Also logged by myapp, then root

Configuration with dictConfig

For production applications, use a dictionary-based configuration:

import logging.config

LOGGING_CONFIG = {
    "version": 1,
    "formatters": {
        "standard": {
            "format": "%(asctime)s [%(levelname)s] %(name)s: %(message)s"
        },
        "detailed": {
            "format": "%(asctime)s [%(levelname)s] %(name)s:%(lineno)d: %(message)s"
        },
    },
    "handlers": {
        "console": {
            "class": "logging.StreamHandler",
            "level": "INFO",
            "formatter": "standard",
        },
        "file": {
            "class": "logging.handlers.RotatingFileHandler",
            "level": "DEBUG",
            "formatter": "detailed",
            "filename": "app.log",
            "maxBytes": 10485760,  # 10MB
            "backupCount": 5,
        },
        "error_file": {
            "class": "logging.handlers.RotatingFileHandler",
            "level": "ERROR",
            "formatter": "detailed",
            "filename": "errors.log",
            "maxBytes": 10485760,
            "backupCount": 3,
        },
    },
    "loggers": {
        "myapp": {
            "level": "DEBUG",
            "handlers": ["console", "file"],
            "propagate": False,
        },
        "myapp.database": {
            "level": "WARNING",
            "handlers": ["error_file"],
        },
    },
    "root": {
        "level": "WARNING",
        "handlers": ["console"],
    },
}

logging.config.dictConfig(LOGGING_CONFIG)

Structured Logging (JSON)

For log aggregation systems like ELK or Loki, JSON-formatted logs are ideal:

import json
import logging

class JSONFormatter(logging.Formatter):
    def format(self, record):
        log_entry = {
            "timestamp": self.formatTime(record),
            "level": record.levelname,
            "logger": record.name,
            "message": record.getMessage(),
            "module": record.module,
            "line": record.lineno,
        }
        if hasattr(record, "extra_data"):
            log_entry["extra"] = record.extra_data
        if record.exc_info and record.exc_info[0]:
            log_entry["exception"] = self.formatException(record.exc_info)
        return json.dumps(log_entry)

logger = logging.getLogger("myapp")
handler = logging.StreamHandler()
handler.setFormatter(JSONFormatter())
logger.addHandler(handler)
logger.setLevel(logging.DEBUG)

# With extra context
logger.info("User logged in", extra={"extra_data": {
    "user_id": 42,
    "ip": "192.168.1.1",
    "method": "oauth"
}})

Expected output (JSON):

{"timestamp": "2026-06-29 12:00:00,000", "level": "INFO", "logger": "myapp", "message": "User logged in", "module": "<module>", "line": 10, "extra": {"user_id": 42, "ip": "192.168.1.1", "method": "oauth"}}

Common Pitfalls

Pitfall 1: Multiple Handlers = Duplicate Messages

# If you call basicConfig() AND add handlers, you get duplicates
logging.basicConfig(level=logging.INFO)
logger = logging.getLogger(__name__)
logger.addHandler(logging.StreamHandler())  # BAD: now messages print twice!

Pitfall 2: Not Using name

# BAD: hardcoded logger name
logger = logging.getLogger("myapp")

# GOOD: uses module path — enables hierarchy
logger = logging.getLogger(__name__)
# For myapp/database.py, __name__ is "myapp.database"

Pitfall 3: Logging Sensitive Data

# NEVER log passwords, tokens, or PII
logger.info(f"User logged in: {username}:{password}")  # SECURITY RISK!
logger.info(f"Credit card: {card_number}")  # PCI violation!

# Log safely
logger.info(f"User logged in: {username}")
logger.info(f"Credit card: ****{card_number[-4:]}")

Practice Questions

  1. Configure logging to write WARNING and above to stderr, and everything to a file.

  2. Create a custom handler that sends ERROR and CRITICAL logs to a Webhook URL.

  3. Write a decorator that logs function calls with arguments and return values at DEBUG level.

  4. Implement log sampling: log only every 10th DEBUG message to reduce volume.

  5. Create a logger Adapter that automatically adds user_id and request_id to every log entry.

Challenge: Log Monitor

Write a class that watches a log file (like tail -f) and alerts when:

  • More than 5 ERROR messages appear in 60 seconds
  • A CRITICAL message appears
  • More than 100 WARNING messages appear in 5 minutes

Use rotating file handling and implement debouncing (don't alert more than once per minute per condition). DodaTech's monitoring system uses this exact pattern to detect security incidents in real-time.

Real-World Task: Audit Logging

DodaTech applications need audit logs for Compliance. Create a logging setup that:

  1. Logs all user actions (login, logout, file access, config changes) to a JSON audit log
  2. Audit log is in a separate file, never rotated (append-only)
  3. Regular application logs go to a rotating file
  4. Security events (failed logins, permission errors) go to a separate security log
  5. Each log entry includes: timestamp, user_id, action, resource, status, ip_address
← Previous Python CSV, JSON, and YAML — Complete Guide Next → Python Dates and Times — datetime Module Complete Guide

Built by the developers of DodaTech

Doda Browser, DodaZIP & Durga Antivirus Pro

Home Browse Python