Allowed File Types — Complete Guide
In this tutorial, you will learn about Allowed File Types. We cover key concepts, practical examples, and best practices to help you master this topic.
Allowed file type restrictions prevent users from uploading dangerous file formats like executables, scripts, or HTML files that could compromise server security.
What You'll Learn
By the end of this lesson, you will understand why file extension checks are unreliable, how to detect file types by inspecting binary signatures, and how to implement a whitelist-based type validation system.
Why It Matters
Attackers commonly upload executable files disguised as images. A server that trusts the file extension or Content-Type header can be tricked into running arbitrary code. Validating the actual file bytes is the only reliable defense.
Real-World Use
A SaaS document editor accepts only PDF, DOCX, and XLSX files. The server checks the file header magic bytes before processing, rejecting ZIP bombs disguised as PDFs and executables renamed to .pdf.
File Type Validation Layers
flowchart LR
File[Uploaded File] --> Ext[Extension Check]
Ext --> MIME[MIME Type Check]
MIME --> Magic[Magic Byte Check]
Magic --> Content[Content Inspection]
Content -->|Pass| Accept[Accept File]
Content -->|Fail| Reject[Reject File]
Extension Whitelist
The simplest check uses the file extension. It is also the easiest to bypass. An attacker can rename virus.exe to virus.jpg and pass the extension check.
# extension_check.py
from typing import List, Optional
import os
ALLOWED_EXTENSIONS = {".jpg", ".jpeg", ".png", ".gif", ".pdf", ".docx"}
def check_extension(filename: str) -> Optional[str]:
ext = os.path.splitext(filename)[1].lower()
if ext not in ALLOWED_EXTENSIONS:
allowed = ", ".join(sorted(ALLOWED_EXTENSIONS))
return f"Extension '{ext}' not allowed. Allowed: {allowed}"
return None
test_files = [
"photo.jpg",
"document.pdf",
"script.exe",
"README.md",
"image.PNG",
"virus.exe.jpg",
]
for f in test_files:
error = check_extension(f)
if error:
print(f" REJECTED: {f} -> {error}")
else:
print(f" ACCEPTED: {f}")
Expected output:
ACCEPTED: photo.jpg
ACCEPTED: document.pdf
REJECTED: script.exe -> Extension '.exe' not allowed. Allowed: .docx, .gif, .jpeg, .jpg, .pdf, .png
REJECTED: README.md -> Extension '.md' not allowed. Allowed: .docx, .gif, .jpeg, .jpg, .pdf, .png
ACCEPTED: image.PNG
ACCEPTED: virus.exe.jpg
Notice that virus.exe.jpg passes the extension check because its last extension is .jpg. Extension checks alone are not safe.
Magic Byte Detection
Every file format starts with a unique sequence of bytes called magic bytes or file signatures. Checking these bytes is far more reliable than checking extensions.
# magic_bytes.py
from typing import Dict, Optional, Tuple
# Common file signatures (first bytes)
MAGIC_SIGNATURES: Dict[str, bytes] = {
"image/jpeg": b"\xff\xd8\xff",
"image/png": b"\x89PNG\r\n\x1a\n",
"image/gif": b"GIF8",
"application/pdf": b"%PDF",
"application/zip": b"PK\x03\x04",
"application/vnd.openxmlformats-officedocument.wordprocessingml.document": b"PK\x03\x04",
"text/plain": None,
}
def detect_mime_type(data: bytes) -> Optional[str]:
for mime, sig in MAGIC_SIGNATURES.items():
if sig and data.startswith(sig):
return mime
return "application/octet-stream"
def validate_file_type(data: bytes, allowed_mimes: set) -> Tuple[bool, str]:
detected = detect_mime_type(data)
if detected and detected in allowed_mimes:
return True, detected
return False, detected or "unknown"
samples = [
(b"\xff\xd8\xff\xe0\x00\x10JFIF", "photo.jpg"),
(b"%PDF-1.4\n%\xFF\xFF\xFF\xFF", "doc.pdf"),
(b"MZ\x90\x00\x03\x00\x00\x00", "notavirus.exe"),
(b"GIF89a\x00\x00\x00\x00", "animation.gif"),
(b"\x89PNG\r\n\x1a\n\x00\x00\x00\rIHDR", "image.png"),
]
allowed = {"image/jpeg", "image/png", "image/gif", "application/pdf"}
for data, name in samples:
valid, mime = validate_file_type(data, allowed)
status = "ACCEPTED" if valid else "REJECTED"
print(f" {status}: {name} -> {mime}")
Expected output:
ACCEPTED: photo.jpg -> image/jpeg
ACCEPTED: doc.pdf -> application/pdf
REJECTED: notavirus.exe -> application/octet-stream
ACCEPTED: animation.gif -> image/gif
ACCEPTED: image.png -> image/png
Whitelist vs Blacklist
Never use a blacklist approach. Blacklists are incomplete and easily bypassed. A new dangerous extension or format bypasses the blacklist until it is added.
# whitelist_vs_blacklist.py
from typing import List, Set
BLACKLIST = {".exe", ".bat", ".sh", ".ps1", ".vbs", ".scr"}
WHITELIST = {".jpg", ".jpeg", ".png", ".gif", ".pdf", ".docx", ".xlsx", ".txt"}
def blacklist_check(ext: str) -> bool:
return ext.lower() not in BLACKLIST
def whitelist_check(ext: str) -> bool:
return ext.lower() in WHITELIST
dangers = [
".exe",
".EXE",
".ExE",
".hta",
".vbe",
".jar",
".php",
".html",
".svg",
]
print(f"{'Extension':12s} {'Blacklist':12s} {'Whitelist':12s}")
print("-" * 36)
for ext in dangers:
b = "ALLOW" if blacklist_check(ext) else "BLOCK"
w = "ALLOW" if whitelist_check(ext) else "BLOCK"
print(f"{ext:12s} {b:12s} {w:12s}")
Expected output:
Extension Blacklist Whitelist
------------------------------------
.exe BLOCK BLOCK
.EXE ALLOW BLOCK
.ExE ALLOW BLOCK
.hta ALLOW BLOCK
.vbe ALLOW BLOCK
.jar ALLOW BLOCK
.php ALLOW BLOCK
.html ALLOW BLOCK
.svg ALLOW BLOCK
The blacklist is case-sensitive and misses .hta, .vbe, .jar, .php, .html, and .svg. The whitelist blocks everything not explicitly allowed.
Combined Validator
A robust file type validator checks the extension, the declared MIME type, and the magic bytes, and rejects if any layer fails.
# combined_validator.py
import os
from typing import Dict, Optional, Tuple
MAGIC_MAP: Dict[str, bytes] = {
"image/jpeg": b"\xff\xd8\xff",
"image/png": b"\x89PNG",
"image/gif": b"GIF8",
"application/pdf": b"%PDF",
}
EXTENSION_MIME_MAP: Dict[str, str] = {
".jpg": "image/jpeg",
".jpeg": "image/jpeg",
".png": "image/png",
".gif": "image/gif",
".pdf": "application/pdf",
}
def validate_upload(filename: str, declared_mime: str, data: bytes) -> Tuple[bool, str]:
ext = os.path.splitext(filename)[1].lower()
expected_mime = EXTENSION_MIME_MAP.get(ext)
if not expected_mime:
return False, f"Extension {ext} not allowed"
if declared_mime != expected_mime:
return False, f"MIME mismatch: declared {declared_mime}, expected {expected_mime}"
magic = MAGIC_MAP.get(expected_mime)
if magic and not data.startswith(magic):
detected_mime = "unknown"
for mime, sig in MAGIC_MAP.items():
if data.startswith(sig):
detected_mime = mime
break
return False, f"Magic bytes mismatch: expected {expected_mime}, detected {detected_mime}"
return True, expected_mime
test_cases = [
("photo.jpg", "image/jpeg", b"\xff\xd8\xff\xe0"),
("fake.pdf", "application/pdf", b"\xff\xd8\xff\xe0"),
("evil.exe", "image/jpeg", b"\xff\xd8\xff\xe0"),
("doc.pdf", "image/png", b"%PDF-1.4"),
("script.txt", "text/plain", b"#!/bin/bash"),
]
for name, mime, data in test_cases:
valid, msg = validate_upload(name, mime, data)
status = "PASS" if valid else "FAIL"
print(f" {status}: {name:12s} {msg}")
Expected output:
PASS: photo.jpg image/jpeg
FAIL: fake.pdf Magic bytes mismatch: expected application/pdf, detected image/jpeg
FAIL: evil.exe Extension .exe not allowed
FAIL: doc.pdf MIME mismatch: declared image/png, expected application/pdf
FAIL: script.txt Extension .txt not allowed
Common Mistakes
1. Using a Blacklist Instead of a Whitelist
Blacklists are incomplete and case-sensitive. Whitelists explicitly define what is allowed and reject everything else.
2. Only Checking the Extension
Extensions are easily renamed. A dangerous file with a .pdf extension is still dangerous.
3. Trusting the Content-Type Header
The Content-Type header is set by the client and can be arbitrary. Validate actual file content.
4. Forgetting Case Sensitivity
.JPG, .Jpg, and .jpg should all be treated the same. Normalize extensions to lowercase.
5. Allowing SVG Uploads
SVG files can contain JavaScript and are a common XSS vector. Treat SVGs as untrusted if you must allow them.
Practice Questions
1. What are magic bytes?
The first few bytes of a file that uniquely identify its format (e.g., %PDF for PDF files).
2. Why is a whitelist safer than a blacklist?
A whitelist explicitly allows only known-safe types. A blacklist must be continuously updated as new threats emerge.
3. Can magic byte detection be fooled?
An attacker can prepend valid magic bytes to a malicious file. However, the file will still be detected by content inspection and antivirus scanning.
4. What is the risk of allowing SVG uploads?
SVGs can contain embedded JavaScript that executes in the browser, leading to cross-site scripting (XSS).
Challenge
Build a file type detector that can identify 15 common file formats by their magic bytes. Include office documents, archives, images, and audio formats.
FAQ
Mini Project: Magic Byte Database
# magic_database.py
import os
from typing import Dict, Optional, Tuple
class MagicDatabase:
def __init__(self):
self.signatures: Dict[str, bytes] = {}
self.extensions: Dict[str, str] = {}
def register(self, mime: str, magic: bytes, extensions: list):
self.signatures[mime] = magic
for ext in extensions:
self.extensions[ext] = mime
def identify(self, data: bytes, filename: str = "") -> Tuple[str, float]:
ext = os.path.splitext(filename)[1].lower() if filename else ""
for mime, sig in self.signatures.items():
if data.startswith(sig):
return mime, 1.0
if ext in self.extensions:
return self.extensions[ext], 0.5
return "application/octet-stream", 0.0
db = MagicDatabase()
db.register("image/jpeg", b"\xff\xd8\xff", [".jpg", ".jpeg"])
db.register("image/png", b"\x89PNG", [".png"])
db.register("application/pdf", b"%PDF", [".pdf"])
db.register("application/zip", b"PK\x03\x04", [".zip"])
db.register("image/gif", b"GIF8", [".gif"])
samples = [
(b"\xff\xd8\xff\xe0", "photo.jpg"),
(b"PK\x03\x04\x00\x00", "archive.zip"),
(b"\x00\x00\x00\x00", "unknown.bin"),
]
for data, name in samples:
mime, confidence = db.identify(data, name)
print(f" {name:15s} -> {mime:30s} (confidence: {confidence})")
Expected output:
photo.jpg -> image/jpeg (confidence: 1.0)
archive.zip -> application/zip (confidence: 1.0)
unknown.bin -> application/octet-stream (confidence: 0.0)
What's Next
You understand how to restrict file types. Next, learn deep file validation beyond Type Checking, including content integrity and structure validation.
Built by the developers of DodaTech
Doda Browser, DodaZIP & Durga Antivirus Pro