Python Regular Expressions — Complete Guide
In this tutorial, you will learn about Python Regular Expressions. We cover key concepts, practical examples, and best practices to help you master this topic.
Regular expressions (regex) are patterns that match text. They're one of the most powerful tools in a developer's toolkit — used for validation, Parsing, search-and-replace, and data extraction. Python's re module gives you full regex support.
In this tutorial, you'll learn Python regex from the ground up. You'll use these skills in real applications like form validation, log analysis, and text processing — exactly what DodaTech tools do when scanning files for threat patterns.
What You'll Learn
- Basic patterns: literals, character classes, quantifiers
- The re module functions: search(), match(), findall(), finditer()
- Groups and capturing
- Substitution with re.sub()
- Compiling patterns for performance
- Lookahead and lookbehind assertions
- Flags: re.IGNORECASE, re.MULTILINE, re.DOTALL
Why Regex Matters
Without regex, you'd write dozens of string methods to do what a single pattern can do. Log analysis at DodaTech uses regex to detect SQL Injection attempts, suspicious IP patterns, and malware signatures in real-time. Every security engineer needs regex.
Basic Patterns
Let's start with the building blocks:
import re
text = "The price is $49.99 and the discount is 15%"
# Literal match
print(re.search(r"price", text)) # Match object (found)
print(re.search(r"cost", text)) # None (not found)
# Character classes
print(re.findall(r"\d+", text)) # ['49', '99', '15'] — digits
print(re.findall(r"\w+", text)) # Word characters (split by non-word)
print(re.findall(r"\s+", text)) # Whitespace chunks
# Custom character classes
print(re.findall(r"[aeiou]", text)) # All vowels
print(re.findall(r"[^a-zA-Z\s]", text)) # Non-letter, non-space: ['$', '.', '%']
Expected output:
<re.Match object; span=(12, 17), match='price'>
None
['49', '99', '15']
['The', 'price', 'is', '49', '99', 'and', 'the', 'discount', 'is', '15']
[' ', ' ', ' ', ' ', ' ', ' ', ' ', ' ']
['e', 'i', 'e', 'i', 'e']
['$', '.', '%']
Quantifiers and Anchors
Quantifiers control how many times a pattern matches:
text = "color colour gray grey"
# ? makes preceding char optional
print(re.findall(r"colou?r", text)) # ['color', 'colour']
# + means one or more
print(re.findall(r"\d+", "App v2.5.1")) # ['2', '5', '1']
# * means zero or more
print(re.findall(r"ab*", "a ab abb")) # ['a', 'ab', 'abb']
# {min,max} for exact counts
print(re.findall(r"\d{3}", "123 45 6789")) # ['123', '678']
print(re.findall(r"\d{2,4}", "123 45 67890")) # ['123', '45', '6789', '0']
# Anchors: ^ (start) and $ (end)
print(re.findall(r"^The", "The quick brown fox")) # ['The']
print(re.findall(r"fox$", "The quick brown fox")) # ['fox']
print(re.findall(r"^fox", "The quick brown fox")) # [] — fox not at start
Groups and Capturing
Groups let you extract specific parts of a match:
text = "Alice: 30, Bob: 25, Charlie: 35"
# Extract name:age pairs
pattern = r"(\w+): (\d+)"
matches = re.findall(pattern, text)
print(matches) # [('Alice', '30'), ('Bob', '25'), ('Charlie', '35')]
# Named groups for clarity
pattern = r"(?P<name>\w+): (?P<age>\d+)"
for match in re.finditer(pattern, text):
print(f"{match.group('name')} is {match.group('age')} years old")
Expected output:
[('Alice', '30'), ('Bob', '25'), ('Charlie', '35')]
Alice is 30 years old
Bob is 25 years old
Charlie is 35 years old
Substitution with re.sub()
Search and replace using patterns:
text = "My phone is 555-1234 and work is 555-5678"
# Mask phone numbers
masked = re.sub(r"\d{3}-\d{4}", "XXX-XXXX", text)
print(masked) # "My phone is XXX-XXXX and work is XXX-XXXX"
# Using groups in replacement
date = "2026-06-29"
reformatted = re.sub(r"(\d{4})-(\d{2})-(\d{2})", r"\3/\2/\1", date)
print(reformatted) # "29/06/2026"
# Using a function for the replacement
def censor_credit_card(match):
card = match.group()
return "****-****-****-" + card[-4:]
log = "Card: 1234-5678-9012-3456, Amount: $50"
safe_log = re.sub(r"\d{4}-\d{4}-\d{4}-\d{4}", censor_credit_card, log)
print(safe_log) # "Card: ****-****-****-3456, Amount: $50"
Lookahead and Lookbehind
These match based on what follows or precedes, without including it in the match:
text = "Win $100! Pay $50. Price: $200"
# Positive lookahead: match $ only if followed by digits
prices = re.findall(r"\$\d+", text)
print(prices) # ['$100', '$50', '$200']
# Extract digits after $ without including $
amounts = re.findall(r"(?<=\$)\d+", text)
print(amounts) # ['100', '50', '200']
# Negative lookahead: match "cat" not followed by "erpillar"
animals = "cat caterpillar catalog"
print(re.findall(r"cat(?!erpillar)", animals)) # ['cat', 'cat']
# Positive lookbehind: match digits preceded by "Page "
pages = "See Page 42 and Page 100"
print(re.findall(r"(?<=Page )\d+", pages)) # ['42', '100']
Compiling Patterns for Performance
When you use a pattern repeatedly, compile it:
# Slow: pattern recompiled each time
for line in thousands_of_lines:
if re.search(r"\d{3}-\d{2}-\d{4}", line):
print("Found SSN")
# Fast: pattern compiled once
ssn_pattern = re.compile(r"\d{3}-\d{2}-\d{4}")
for line in thousands_of_lines:
if ssn_pattern.search(line):
print("Found SSN")
Real-World: Email Validator
import re
def validate_email(email):
"""Validate an email address format."""
pattern = r"^[a-zA-Z0-9._%+-]+@[a-zA-Z0-9.-]+\.[a-zA-Z]{2,}$"
return bool(re.match(pattern, email))
# Test
emails = ["user@example.com", "invalid@", "user@.com", "user+tag@company.co.uk"]
for e in emails:
result = "valid" if validate_email(e) else "invalid"
print(f"{e}: {result}")
Expected output:
user@example.com: valid
invalid@: invalid
user@.com: invalid
user+tag@company.co.uk: valid
Common Pitfalls
Pitfall 1: Greedy vs Non-Greedy
text = "<h1>Title</h1><p>Text</p>"
# Greedy (default): matches as much as possible
print(re.findall(r"<.*>", text)) # ['<h1>Title</h1><p>Text</p>']
# Non-greedy: matches as little as possible
print(re.findall(r"<.*?>", text)) # ['<h1>', '</h1>', '<p>', '</p>']
Pitfall 2: Forgetting to Escape Special Characters
# Wrong: . matches ANY character
print(re.findall(r"file.txt", "fileXtxt")) # Found! (dot matches X)
# Right: escape the dot
print(re.findall(r"file\.txt", "fileXtxt")) # Not found
Pitfall 3: Raw Strings
Always use raw strings (r"...") — otherwise backslashes get interpreted:
# Without r-string: \b is backspace character!
pattern = "\bword\b" # Wrong!
# With r-string: \b is word boundary
pattern = r"\bword\b" # Correct
Practice Questions
Write a function that extracts all URLs from a string (http/https/ftp).
Validate a US phone number format: (555) 123-4567 or 555-123-4567.
Extract all hashtags from a tweet: "Loving #Python and #regex! #100DaysOfCode".
Find all words that start with a capital letter in a sentence.
Write a regex that matches IPv4 addresses (four numbers 0-255 separated by dots).
Challenge: Log Analyzer
Write a function that parses this syslog line:
Jun 29 12:34:56 server sshd[1234]: Failed password for root from 192.168.1.100 port 22 ssh2
Extract: timestamp, Process name, PID, event type, username, source IP, and port.
Real-World Task: Config Variable Substitution
Write a function that takes a template string with ${VARIABLE} placeholders and a dictionary of values, and replaces each placeholder with its value. Handle the case where a variable is undefined (leave it as-is with a warning). This is used by DodaTech's deployment system to inject environment-specific configuration values.
Built by the developers of DodaTech
Doda Browser, DodaZIP & Durga Antivirus Pro