Python Collections Module — Counter, defaultdict, deque, namedtuple
In this tutorial, you will learn about Python Collections Module. We cover key concepts, practical examples, and best practices to help you master this topic.
Python's standard library includes the collections module — a set of specialized container data types that make common programming tasks easier, faster, and more readable. If you find yourself writing boilerplate dict logic, you probably need a collections class.
In this tutorial, you'll master the five most useful collection types. DodaTech uses these daily for counting threat patterns, building lookup tables, managing job queues, and creating lightweight data objects.
What You'll Learn
- Counter: count elements like a pro
- defaultdict: dict that never raises KeyError
- deque: fast appends/pops from both ends
- namedtuple: readable, immutable data objects
- OrderedDict: dict with insertion order (Python 3.7+ dicts do this natively)
- ChainMap: combine multiple dicts into one view
Why Collections Matter
Python's built-in dict and list handle most cases. But the collections module solves specific problems more elegantly. Without it, you'd write extra if-statements, manual counting loops, or custom classes. With it, your code becomes cleaner and faster.
Counter — Count Anything
Counter automatically counts items in an iterable:
from collections import Counter
# Count characters in a string
text = "mississippi"
counter = Counter(text)
print(counter) # Counter({'i': 4, 's': 4, 'p': 2, 'm': 1})
# Most common items
print(counter.most_common(2)) # [('i', 4), ('s', 4)]
# Count words in a list
words = ["apple", "banana", "apple", "orange", "banana", "apple"]
word_count = Counter(words)
print(word_count["apple"]) # 3
print(word_count["grape"]) # 0 (no KeyError!)
# Update with more data
word_count.update(["apple", "grape"])
print(word_count["apple"]) # 4
Expected output:
Counter({'i': 4, 's': 4, 'p': 2, 'm': 1})
[('i', 4), ('s', 4)]
3
0
4
Real-World: IP Address Counter
Security analysts at DodaTech use Counter to identify suspicious IPs making too many requests:
from collections import Counter
log_entries = [
"192.168.1.1 - GET /index.html",
"192.168.1.2 - GET /login.php",
"192.168.1.1 - POST /login.php",
"192.168.1.1 - GET /admin.php",
"192.168.1.3 - GET /index.html",
"192.168.1.1 - POST /admin.php",
]
# Extract IPs
ips = [entry.split(" - ")[0] for entry in log_entries]
ip_counts = Counter(ips)
# Find potential brute-force attacks
for ip, count in ip_counts.most_common():
if count > 2:
print(f"ALERT: {ip} made {count} requests — possible attack")
Expected output:
ALERT: 192.168.1.1 made 4 requests — possible attack
defaultdict — Safe Default Values
defaultdict returns a default value when a key is missing, instead of raising KeyError:
from collections import defaultdict
# Group items by first letter
words = ["apple", "banana", "avocado", "blueberry", "cherry"]
groups = defaultdict(list)
for word in words:
groups[word[0]].append(word)
print(dict(groups))
# {'a': ['apple', 'avocado'], 'b': ['banana', 'blueberry'], 'c': ['cherry']}
# Without defaultdict, you'd write:
groups = {}
for word in words:
first = word[0]
if first not in groups:
groups[first] = []
groups[first].append(word)
# Count nested data
nested_counts = defaultdict(lambda: defaultdict(int))
nested_counts["2026"]["June"] += 1
nested_counts["2026"]["June"] += 1
nested_counts["2026"]["July"] += 1
print(dict(nested_counts))
# {'2026': defaultdict(<class 'int'>, {'June': 2, 'July': 1})}
deque — Fast Queues from Both Ends
deque (double-ended queue) is optimized for adding/removing from both ends:
from collections import deque
# Create a queue
queue = deque(["task1", "task2", "task3"])
queue.append("task4") # Add to right
queue.appendleft("urgent") # Add to left
print(queue) # deque(['urgent', 'task1', 'task2', 'task3', 'task4'])
# Pop from either end
print(queue.pop()) # 'task4' — from right
print(queue.popleft()) # 'urgent' — from left
# Fixed-size buffer (drops oldest when full)
buffer = deque(maxlen=3)
for i in range(10):
buffer.append(i)
print(list(buffer))
# [0]
# [0, 1]
# [0, 1, 2]
# [1, 2, 3]
# [2, 3, 4]
# ... keeps only last 3
namedtuple — Readable Lightweight Objects
namedtuple creates tuple subclasses with named fields:
from collections import namedtuple
# Define a Point type
Point = namedtuple("Point", ["x", "y"])
p = Point(10, 20)
print(p.x, p.y) # 10 20
print(p[0], p[1]) # 10 20 (still works as tuple)
x, y = p # Tuple unpacking
print(x, y) # 10 20
# Perfect for small data objects
Employee = namedtuple("Employee", "name role salary")
alice = Employee("Alice", "Engineer", 75000)
print(f"{alice.name} is an {alice.role}")
# Convert to dict
print(alice._asdict())
# {'name': 'Alice', 'role': 'Engineer', 'salary': 75000}
ChainMap — Combining Multiple Dicts
ChainMap groups multiple dicts into a single view:
from collections import ChainMap
defaults = {"theme": "dark", "language": "en", "debug": False}
user_prefs = {"language": "fr", "notifications": True}
runtime = {"debug": True}
# Priority: runtime > user_prefs > defaults
config = ChainMap(runtime, user_prefs, defaults)
print(config["theme"]) # "dark" — from defaults
print(config["language"]) # "fr" — from user_prefs (overrides defaults)
print(config["debug"]) # True — from runtime (overrides all)
print(config["notifications"]) # True — from user_prefs
Common Pitfalls
Pitfall 1: mutable default arguments with defaultdict
# WRONG: int() returns 0, not a new list
d = defaultdict(list) # Correct: list factory
d = defaultdict([]) # TypeError: first argument must be callable
# WRONG: lambda with mutable default
d = defaultdict(lambda: []) # This is fine actually
Pitfall 2: Counter with large data
Counter stores everything in memory. For streaming data, use a different approach:
# Good for moderate-sized data
with open("large_file.txt") as f:
counter = Counter(f.read().split())
# For HUGE files, stream instead
from itertools import islice
def stream_count(filename, chunk_size=10000):
counter = Counter()
with open(filename) as f:
while True:
chunk = list(islice(f, chunk_size))
if not chunk:
break
counter.update(" ".join(chunk).split())
return counter
Practice Questions
Use Counter to find the most common IP address in a list of log entries.
Use defaultdict to build an inverted index: given documents {doc1: "apple banana", doc2: "banana cherry"}, produce {word: [doc_ids]}.
Use deque to implement a simple undo buffer that stores at most 10 operations.
Use namedtuple to represent a Stock with fields: symbol, price, volume.
Use ChainMap to merge environment variables (highest priority), config file, and defaults.
Challenge: Cache with LRU Eviction
Implement an LRU (Least Recently Used) cache using deque and dict. When the cache exceeds maxsize, remove the least recently accessed item. Hint: use a deque to track access order, and a dict for fast lookups. This is similar to how functools.lru_cache works internally.
Real-World Task: Port Scanner Report
Write a function that takes a list of (ip, port, status) tuples and produces:
- A Counter of open ports per IP (find which IPs have the most open ports)
- A defaultdict(list) mapping port numbers to IPs (find which ports are most commonly open)
- A deque of the last 10 scan results for live monitoring
DodaTech's network monitoring tools use this exact pattern to identify compromised hosts running unexpected services.
Built by the developers of DodaTech
Doda Browser, DodaZIP & Durga Antivirus Pro