Input Sanitization — Complete Guide
In this tutorial, you'll learn about Input Sanitization. We cover key concepts, practical examples, and best practices to help you understand and apply this topic effectively.
Input sanitization cleans user input by removing or escaping dangerous characters, trimming whitespace, normalizing formats, and stripping unexpected content.
What You'll Learn
By the end of this lesson, you will implement sanitization functions, prevent XSS and SQL Injection through sanitization, and build a sanitization pipeline.
Why It Matters
Validation tells you if data is valid. Sanitization makes it safe. Even valid input can contain XSS payloads, excessive whitespace, or hidden Unicode characters.
Real-World Use
WordPress sanitizes post content before storing: strips disallowed HTML, escapes quotes, removes JavaScript event handlers, and normalizes character encoding.
Sanitization Flow
flowchart LR
Input[Raw Input] --> Trim[Trim Whitespace]
Trim --> Escape[Escape HTML]
Escape --> Strip[Strip Dangerous Chars]
Strip --> Normalize[Normalize Unicode]
Normalize --> Safe[Safe Output]
Sanitization Functions
# sanitization.py
import re
from typing import Any, Dict, List, Optional
class Sanitizer:
@staticmethod
def trim(value: str) -> str:
if not value:
return value
return value.strip()
@staticmethod
def remove_html(value: str) -> str:
if not isinstance(value, str):
return value
return re.sub(r'<[^>]+>', '', value)
@staticmethod
def escape_html(value: str) -> str:
if not isinstance(value, str):
return value
replacements = {
"&": "&",
"<": "<",
">": ">",
'"': """,
"'": "'",
}
for char, escaped in replacements.items():
value = value.replace(char, escaped)
return value
@staticmethod
def strip_non_alphanumeric(value: str, allow_spaces: bool = True) -> str:
pattern = r'[^\w\s]' if allow_spaces else r'[^\w]'
return re.sub(pattern, '', value)
@staticmethod
def truncate(value: str, max_length: int = 1000) -> str:
if not value or len(value) <= max_length:
return value
return value[:max_length] + "..."
sanitizer = Sanitizer()
inputs = [
" Hello World! ",
"<script>alert('xss')</script>",
"Normal text with <b>HTML</b>",
"a<>&\"'chars",
"Long " + "x" * 2000,
]
print(f"Trim: '{sanitizer.trim(inputs[0])}'")
print(f"Strip: '{sanitizer.remove_html(inputs[1])}'")
print(f"Escape: '{sanitizer.escape_html(inputs[3])}'")
print(f"Alpha: '{sanitizer.strip_non_alphanumeric('hello@world!123', True)}'")
print(f"Truncate: '{sanitizer.truncate(inputs[4], 10)}'")
Expected output:
Trim: 'Hello World!'
Strip: 'alert('xss')'
Escape: 'a<>&"'chars'
Alpha: 'helloworld123'
Truncate: 'xxxxxxxxxx...'
Request Body Sanitizer
# body_sanitizer.py
from typing import Any, Dict, List, Optional
class BodySanitizer:
def __init__(self):
self.rules: Dict[str, List[callable]] = {}
def add_rule(self, field: str, sanitizer_fn: callable):
self.rules.setdefault(field, []).append(sanitizer_fn)
def sanitize(self, data: Dict) -> Dict:
result = dict(data)
for field, sanitizers in self.rules.items():
if field in result and isinstance(result[field], str):
for fn in sanitizers:
result[field] = fn(result[field])
return result
sanitizer = BodySanitizer()
sanitizer.add_rule("name", lambda v: v.strip())
sanitizer.add_rule("name", lambda v: v[:100])
sanitizer.add_rule("bio", lambda v: v.strip())
sanitizer.add_rule("bio", lambda v: v.replace("<", "<").replace(">", ">"))
input_data = {
"name": " Alice<script>alert(1)</script> ",
"bio": "Hello <script>attack</script>",
"age": "30",
}
sanitized = sanitizer.sanitize(input_data)
print(f"Name: '{sanitized['name']}'")
print(f"Bio: '{sanitized['bio']}'")
print(f"Age: '{sanitized['age']}' (unchanged)")
Expected output:
Name: 'Alice<script>alert(1)</script>'
Bio: 'Hello <script>attack</script>'
Age: '30' (unchanged)
Unicode Normalization
# unicode_sanitize.py
import unicodedata
from typing import Any, Dict, List
class UnicodeSanitizer:
@staticmethod
def normalize(value: str, form: str = "NFKC") -> str:
return unicodedata.normalize(form, value)
@staticmethod
def strip_control_chars(value: str) -> str:
return ''.join(c for c in value if unicodedata.category(c)[0] != 'C')
@staticmethod
def detect_homoglyph(value: str) -> bool:
suspicious = {
'A': 'А', 'B': 'В', 'C': 'С', 'E': 'Е', 'H': 'Н',
'I': 'І', 'K': 'К', 'M': 'М', 'O': 'О', 'P': 'Р',
}
for c in value:
if ord(c) > 127 and any(
unicodedata.name(c, '').startswith(unicodedata.name(l, 'X')[:2])
for l in suspicious.values()
):
return True
return False
us = UnicodeSanitizer()
inputs = [
"caf\u00e9", # composed: café
"cafe\u0301", # decomposed: cafe + combining accent
"\u0000Hello\u0007",
]
for inp in inputs:
norm = us.normalize(inp)
clean = us.strip_control_chars(norm)
print(f"Original: {inp!r} -> Normalized: {norm!r} -> Clean: {clean!r}")
homoglyph_test = "Hеllo" # Cyrillic 'е' instead of Latin 'e'
print(f"Homoglyph detected: {us.detect_homoglyph(homoglyph_test)}")
Expected output:
Original: 'caf\xe9' -> Normalized: 'caf\xe9' -> Clean: 'caf\xe9'
Original: 'cafe\u0301' -> Normalized: 'caf\xe9' -> Clean: 'caf\xe9'
Original: '\x00Hello\x07' -> Normalized: '\x00Hello\x07' -> Clean: 'Hello'
Homoglyph detected: True
Common Mistakes
1. Sanitizing Without Validating
Sanitization is not validation. A sanitized email is still invalid if it lacks @. Validate then sanitize.
2. Double Sanitization
Sanitizing twice can corrupt data. Keep track of what has been sanitized.
3. Not Trimming Input
Leading/trailing whitespace causes comparison failures. Always trim strings.
4. Over-Sanitization
Removing characters that are valid in some contexts. Stripping all special characters may break multilingual input.
5. Ignoring Unicode Attacks
Homoglyph attacks use lookalike Unicode characters. Normalize to NFKC to prevent bypass.
Practice Questions
1. What is the difference between validation and sanitization?
Validation checks if data is correct. Sanitization cleans data to make it safe.
2. What is HTML escaping?
Replacing < > & " ' with their HTML entity equivalents to prevent XSS.
3. Why trim whitespace?
Leading/trailing spaces cause comparison failures, storage waste, and display issues.
4. What is Unicode normalization?
Converting Unicode text to a standard form (NFC/NFKC) to prevent bypass using composed vs decomposed characters.
Challenge
Build a comprehensive sanitizer for a comment form: strip HTML, escape remaining entities, trim, truncate to 1000 chars, normalize Unicode, and detect homoglyph attacks.
FAQ
Mini Project: Sanitization Pipeline
# sanitize_pipeline.py
from typing import Any, Callable, Dict, List
class SanitizePipeline:
def __init__(self):
self.steps: List[Callable] = []
def add(self, step: Callable):
self.steps.append(step)
def run(self, data: Dict) -> Dict:
result = dict(data)
for step in self.steps:
result = step(result)
return result
def trim_strings(data: Dict) -> Dict:
return {k: v.strip() if isinstance(v, str) else v for k, v in data.items()}
def escape_html(data: Dict) -> Dict:
return {k: v.replace("<", "<").replace(">", ">") if isinstance(v, str) else v for k, v in data.items()}
pipeline = SanitizePipeline()
pipeline.add(trim_strings)
pipeline.add(escape_html)
result = pipeline.run({"name": " <b>Alice</b> "})
print(f"Sanitized: {result}")
Expected output:
Sanitized: {'name': '<b>Alice</b>'}
What's Next
You understand sanitization. Next, learn type coercion, then custom validators.
Built by the developers of DodaTech
Doda Browser, DodaZIP & Durga Antivirus Pro