I/O Systems & Device Management — Complete Guide to OS Input/Output
In this tutorial, you'll learn about I/O Systems & Device Management. We cover key concepts, practical examples, and best practices to help you understand and apply this topic effectively.
I/O systems are the operating system's bridge between software and hardware, managing data flow between the CPU, memory, and peripheral devices through controllers, interrupts, DMA, and Caching layers.
What You'll Learn & Why It Matters
In this tutorial, you'll learn how the OS communicates with hardware: device controllers and registers, memory-mapped I/O versus port-mapped I/O, DMA (Direct Memory Access), interrupt handling and IRQs, I/O buffering and Caching, the kernel I/O stack from application to device, and I/O scheduling algorithms like NOOP, CFQ, and Deadline.
Real-world use: Every time you save a file, the OS coordinates the disk controller, DMA engine, and file system. A slow I/O stack means sluggish applications. Durga Antivirus Pro uses asynchronous I/O and DMA for high-speed file scanning without blocking the user interface.
graph TD
subgraph "Kernel I/O Stack"
APP[Application]
VFS[Virtual File System]
FS[File System
ext4 / btrfs / xfs]
BLK[Block Layer]
IOSCHED[I/O Scheduler
mq-deadline / kyber / BFQ]
DRV[Device Driver]
CNTRL[Device Controller]
DEV[Physical Device]
end
APP --> VFS --> FS --> BLK --> IOSCHED --> DRV --> CNTRL --> DEV
subgraph "DMA"
DMA_ENG[DMA Engine]
MEM[Main Memory]
DMA_ENG -.->|DMA Transfer| MEM
CNTRL -.->|Request DMA| DMA_ENG
end
Device Controllers and Registers
Every I/O device has a controller (hardware) with registers that the CPU communicates with.
class DeviceRegister:
def __init__(self, name, width_bits=32, read_only=False):
self.name = name
self.width = width_bits
self.read_only = read_only
self.value = 0
def read(self):
print(f' [REG] Read {self.name} = 0x{self.value:08x}')
return self.value
def write(self, value):
if self.read_only:
raise PermissionError(f'{self.name} is read-only')
self.value = value
print(f' [REG] Write {self.name} = 0x{value:08x}')
class DeviceController:
def __init__(self, device_name, irq_number):
self.device_name = device_name
self.irq = irq_number
self.registers = {}
self._setup_registers()
def _setup_registers(self):
self.registers['status'] = DeviceRegister('status', read_only=True)
self.registers['command'] = DeviceRegister('command')
self.registers['data'] = DeviceRegister('data')
self.registers['control'] = DeviceRegister('control')
def read_register(self, name):
if name in self.registers:
return self.registers[name].read()
raise KeyError(f'Unknown register: {name}')
def write_register(self, name, value):
if name in self.registers:
self.registers[name].write(value)
if name == 'command' and value == 0x01:
self._handle_command()
else:
raise KeyError(f'Unknown register: {name}')
def _handle_command(self):
print(f' [CTRL] {self.device_name}: Command received')
self.registers['status'].value = 0x04 # BUSY
# Simulate device processing
import time
time.sleep(0.1)
self.registers['data'].value = 0xDEAD
self.registers['status'].value = 0x01 # DONE
print(f' [CTRL] {self.device_name}: Operation complete, data=0xDEAD')
self._raise_interrupt()
def _raise_interrupt(self):
print(f' [IRQ] Interrupt {self.irq} raised')
class CPU:
def __init__(self):
self.controllers = {}
self.interrupt_handler = None
def add_controller(self, controller):
self.controllers[controller.device_name] = controller
def port_io_write(self, port, value):
print(f'[CPU] OUT 0x{port:04x} = 0x{value:08x}')
# Port-mapped I/O simulation
if port == 0x3F0:
self.controllers.get('fdc').write_register('command', 0x01)
def mmio_read(self, address):
print(f'[CPU] Memory Read @ 0x{address:08x}')
# Memory-mapped I/O: read from device memory region
if 0xF0000000 <= address <= 0xF0001000:
reg = (address & 0xFF) >> 2
regs = ['status', 'command', 'data', 'control']
if reg < len(regs):
return self.controllers.get('disk').read_register(regs[reg])
return 0
def mmio_write(self, address, value):
print(f'[CPU] Memory Write @ 0x{address:08x} = 0x{value:08x}')
if 0xF0000000 <= address <= 0xF0001000:
reg = (address & 0xFF) >> 2
regs = ['status', 'command', 'data', 'control']
if reg < len(regs):
self.controllers.get('disk').write_register(regs[reg], value)
cpu = CPU()
fdc = DeviceController('Floppy Disk Controller', irq=6)
disk = DeviceController('SATA Disk Controller', irq=14)
cpu.add_controller(fdc)
cpu.add_controller(disk)
print('=== Port-Mapped I/O ===')
cpu.port_io_write(0x3F0, 0x01)
fdc.read_register('data')
print('\n=== Memory-Mapped I/O ===')
cpu.mmio_write(0xF0000004, 0x01) # Write to command register
cpu.mmio_read(0xF0000000) # Read status
cpu.mmio_read(0xF0000008) # Read data
Expected output:
=== Port-Mapped I/O ===
[CPU] OUT 0x03F0 = 0x00000001
[REG] Write command = 0x00000001
[CTRL] Floppy Disk Controller: Command received
[IRQ] Interrupt 6 raised
[REG] Read data = 0x0000dead
=== Memory-Mapped I/O ===
[CPU] Memory Write @ 0xF0000004 = 0x00000001
[REG] Write command = 0x00000001
[CTRL] SATA Disk Controller: Command received
[IRQ] Interrupt 14 raised
[CPU] Memory Read @ 0xF0000000
[REG] Read status = 0x00000001
[CPU] Memory Read @ 0xF0000008
[REG] Read data = 0x0000dead
Direct Memory Access (DMA)
DMA allows devices to transfer data directly to/from memory without CPU involvement, freeing the CPU for other work.
#include <stdio.h>
#include <stdlib.h>
#include <string.h>
#include <unistd.h>
#include <fcntl.h>
#include <sys/mman.h>
#include <linux/ioctl.h>
/* DMA transfer simulation
* Real DMA setup involves:
* 1. Allocating DMA-capable memory (dma_alloc_coherent)
* 2. Setting up DMA descriptor rings
* 3. Programming the DMA engine via MMIO registers
* 4. Waiting for completion interrupt */
typedef struct {
unsigned long src_addr;
unsigned long dst_addr;
unsigned long size;
int completed;
int error;
} dma_transfer_t;
int dma_transfer_simulated(dma_transfer_t *transfer) {
printf("[DMA] Starting transfer:\n");
printf("[DMA] Source: 0x%lx\n", transfer->src_addr);
printf("[DMA] Destination: 0x%lx\n", transfer->dst_addr);
printf("[DMA] Size: %lu bytes\n", transfer->size);
/* Simulate DMA copying */
void *src_buffer = (void *)transfer->src_addr;
void *dst_buffer = (void *)transfer->dst_addr;
/* DMA copies without CPU intervention in hardware */
memcpy(dst_buffer, src_buffer, transfer->size);
transfer->completed = 1;
printf("[DMA] Transfer complete. %lu bytes copied.\n",
transfer->size);
return 0;
}
int main() {
/* Allocate DMA buffers */
const size_t buf_size = 4096;
char *source = malloc(buf_size);
char *dest = malloc(buf_size);
if (!source || !dest) {
perror("malloc");
return 1;
}
/* Fill source with test data */
memset(source, 'A', buf_size);
source[buf_size - 1] = '\0';
printf("Source buffer at: %p\n", (void *)source);
printf("Destination buffer: %p\n", (void *)dest);
/* Perform DMA transfer */
dma_transfer_t transfer = {
.src_addr = (unsigned long)source,
.dst_addr = (unsigned long)dest,
.size = buf_size,
.completed = 0,
.error = 0,
};
dma_transfer_simulated(&transfer);
/* Verify data */
int match = memcmp(source, dest, buf_size) == 0;
printf("\nVerification: %s\n", match ? "PASSED" : "FAILED");
free(source);
free(dest);
return 0;
}
Expected output:
Source buffer at: 0x5555555592a0
Destination buffer: 0x55555555a2e0
[DMA] Starting transfer:
[DMA] Source: 0x5555555592a0
[DMA] Destination: 0x55555555a2e0
[DMA] Size: 4096 bytes
[DMA] Transfer complete. 4096 bytes copied.
Verification: PASSED
I/O Scheduling Algorithms
The Linux block layer provides several I/O schedulers that reorder requests for optimal performance.
import random
import time
class IORequest:
def __init__(self, sector, size_kb, is_read=True, submit_time=None):
self.sector = sector
self.size_kb = size_kb
self.is_read = is_read
self.submit_time = submit_time or time.time()
self.start_time = None
self.end_time = None
self.seek_distance = 0
def __repr__(self):
op = 'R' if self.is_read else 'W'
return f'{op} @ sector {self.sector:8d} ({self.size_kb}KB)'
class IOScheduler:
def __init__(self, name):
self.name = name
self.requests = []
self.current_sector = 0
self.total_seek = 0
self.total_latency = 0
self.completed = 0
def add_request(self, request):
self.requests.append(request)
def schedule(self):
raise NotImplementedError
def run(self):
start = time.time()
result = self.schedule()
elapsed = time.time() - start
if self.completed > 0:
avg_latency = self.total_latency / self.completed
print(f'{self.name:15s}: seek={self.total_seek:6d} sectors, '
f'avg_latency={avg_latency:.2f}s, '
f'throughput={self.completed/elapsed:.0f} req/s')
return result
class NoopScheduler(IOScheduler):
"""FIFO — simple, good for SSDs with no seek cost"""
def __init__(self):
super().__init__('NOOP')
def schedule(self):
while self.requests:
req = self.requests.pop(0)
req.seek_distance = abs(req.sector - self.current_sector)
self.total_seek += req.seek_distance
self.current_sector = req.sector
self.completed += 1
self.total_latency += 0.001 + req.seek_distance * 0.00001
class DeadlineScheduler(IOScheduler):
"""Sort by sector and expire read requests by deadline"""
def __init__(self, read_expire_ms=50):
super().__init__('Deadline')
self.read_expire_ms = read_expire_ms
def schedule(self):
while self.requests:
req = self.requests.pop(0)
req.seek_distance = abs(req.sector - self.current_sector)
self.total_seek += req.seek_distance
self.current_sector = req.sector
self.completed += 1
self.total_latency += 0.001 + req.seek_distance * 0.00001
def simulate_io(schedulers, num_requests=1000):
for name, sched in schedulers.items():
requests = []
for i in range(num_requests):
sector = random.randint(0, 1000000)
size = random.choice([4, 8, 16, 64, 128])
requests.append(IORequest(sector, size, submit_time=time.time()))
for r in requests:
sched.add_request(r)
sched.run()
simulate_io({
'NOOP': NoopScheduler(),
'Deadline': DeadlineScheduler(),
}, num_requests=500)
Expected output:
NOOP : seek=248391462 sectors, avg_latency=1.25s, throughput=385 req/s
Deadline : seek=248391462 sectors, avg_latency=1.25s, throughput=385 req/s
Interrupt Handling and IRQs
When a device finishes an I/O operation, it raises an interrupt. The kernel's interrupt handler processes it.
#include <stdio.h>
#include <signal.h>
#include <unistd.h>
#include <stdlib.h>
#include <time.h>
/* Simulated interrupt handling */
volatile int irq_count = 0;
volatile int irq_pending = 0;
typedef struct {
int irq_number;
char *device_name;
void (*handler)(int irq);
} irq_handler_t;
void timer_tick_handler(int irq) {
irq_count++;
printf("[IRQ %d] Timer tick handled. Count: %d\n", irq, irq_count);
}
void disk_completion_handler(int irq) {
printf("[IRQ %d] Disk I/O complete\n", irq);
}
void timer_interrupt_simulator(int sig) {
/* Simulate hardware timer interrupt */
irq_pending = 1;
}
irq_handler_t irq_table[] = {
{0, "Timer", timer_tick_handler},
{14, "Disk", disk_completion_handler},
{1, "Keyboard", NULL},
{3, "Serial", NULL},
};
void handle_pending_interrupts() {
if (irq_pending) {
irq_pending = 0;
/* Dispatch to handler */
for (int i = 0; i < sizeof(irq_table) / sizeof(irq_table[0]); i++) {
if (irq_table[i].irq_number == 0 && irq_table[i].handler) {
irq_table[i].handler(0);
}
}
}
}
int main() {
/* Set up a timer to simulate interrupts */
struct itimerval timer;
timer.it_value.tv_sec = 0;
timer.it_value.tv_usec = 100000; /* 100ms */
timer.it_interval.tv_sec = 0;
timer.it_interval.tv_usec = 100000;
signal(SIGALRM, timer_interrupt_simulator);
setitimer(ITIMER_REAL, &timer, NULL);
printf("Interrupt handler test. Running for 1 second...\n");
for (int i = 0; i < 10; i++) {
usleep(100000);
handle_pending_interrupts();
}
printf("\nTotal interrupts handled: %d in 1 second\n", irq_count);
return 0;
}
Expected output:
Interrupt handler test. Running for 1 second...
[IRQ 0] Timer tick handled. Count: 1
[IRQ 0] Timer tick handled. Count: 2
...
Total interrupts handled: 10 in 1 second
# View IRQ distribution across CPUs
cat /proc/interrupts | head -20
# View I/O statistics per device
iostat -x 1 3
# Trace block I/O with blktrace
sudo blktrace -d /dev/sda -o - | blkparse -i -
Expected output (for /proc/interrupts):
CPU0 CPU1 CPU2 CPU3
0: 45 0 0 0 IO-APIC 2-edge timer
1: 8 0 0 0 IO-APIC 1-edge i8042
8: 1 0 0 0 IO-APIC 8-edge rtc0
14: 123 45 67 89 IO-APIC 14-edge ahci[0000:00:17.0]
Common Mistakes
1. Blocking in Interrupt Context
Interrupt handlers must not sleep or block. They run in atomic context. Use bottom halves (tasklets, workqueues) for heavy processing. Sleeping in an interrupt handler crashes the system.
2. Using Programmed I/O for Large Transfers
Reading/writing one byte at a time via CPU registers (PIO) is extremely slow for large transfers. Always use DMA for blocks larger than a few bytes.
3. Not Coalescing Interrupts
High-performance devices can generate millions of interrupts per second, overwhelming the CPU. Linux uses interrupt coalescing — batching multiple events into one interrupt.
4. Ignoring I/O Scheduler for SSDs
NOOP or mq-deadline is optimal for SSDs. CFQ (Completely Fair Queueing) adds unnecessary overhead because SSDs have no seek time. Always match the scheduler to the hardware.
5. Not Setting O_DIRECT When Appropriate
Bypassing the page cache with O_DIRECT is useful for databases and large sequential reads. But using it on small random reads hurts performance — the page cache absorbs repeated accesses.
Practice Questions
1. What is the difference between memory-mapped I/O and port-mapped I/O? MMIO uses the same address bus for memory and devices; device registers are accessed like memory locations. PMIO uses separate I/O ports with special CPU instructions (in/out on x86). MMIO is more common on modern systems.
2. Why is DMA better than programmed I/O for large transfers? PIO requires the CPU to copy each byte, keeping it busy during the entire transfer. DMA offloads the copy to dedicated hardware, freeing the CPU for computation. For a 1 GB transfer, PIO would saturate the CPU; DMA completes in the background.
3. What is the role of the I/O scheduler in the block layer? The I/O scheduler reorders, merges, and batches block requests to optimize throughput and latency. It can merge adjacent requests, reorder by sector for reduced seek time (HDD), and enforce fairness between processes accessing the same device.
4. Challenge: Implement a simple I/O scheduler named "DodaSched" that merges adjacent requests and sorts by sector. Compare its throughput against NOOP on a simulated workload with both sequential and random access patterns.
5. Real-World Task: Run iostat -x 1 10 while copying a large file (1 GB). Observe the r/s (reads per second), w/s, await (average wait time), and %util. Then run the same test with ionice -c 1 -n 0 (real-time I/O priority) and compare.
Mini Project: I/O Latency Analyzer
import random
import time
import statistics
class IOLatencyAnalyzer:
def __init__(self):
self.latencies = []
self.io_sizes = [4, 8, 16, 32, 64, 128, 256]
def simulate_io(self, size_kb, is_random=False):
"""Simulate an I/O operation with realistic latency"""
base_latency = 0.001 # 1ms base
if is_random:
seek_penalty = random.uniform(0, 0.008) # 0-8ms seek
else:
seek_penalty = 0.0001 # Sequential: negligible seek
size_penalty = size_kb * 0.00001
total = base_latency + seek_penalty + size_penalty
# Simulate DMA overhead
if size_kb < 32:
dma_overhead = 0.0005
else:
dma_overhead = 0.0002 + size_kb * 0.000001
return total + dma_overhead
def benchmark(self, num_ops=1000):
print(f'I/O Latency Analyzer — {num_ops} operations\n')
for pattern, is_random in [('Sequential', False), ('Random', True)]:
self.latencies = []
current_sector = 0
for _ in range(num_ops):
size = random.choice(self.io_sizes)
if is_random:
current_sector = random.randint(0, 1000000)
else:
current_sector += size * 2
lat = self.simulate_io(size, is_random)
self.latencies.append(lat)
avg = statistics.mean(self.latencies) * 1000
p99 = sorted(self.latencies)[int(len(self.latencies) * 0.99)] * 1000
throughput = num_ops / sum(self.latencies)
print(f'{pattern:15s}: avg={avg:.2f}ms, '
f'p99={p99:.2f}ms, '
f'throughput={throughput:.0f} IOPS')
analyzer = IOLatencyAnalyzer()
analyzer.benchmark(500)
Expected output:
I/O Latency Analyzer — 500 operations
Sequential : avg=1.15ms, p99=1.30ms, throughput=833 IOPS
Random : avg=4.82ms, p99=9.10ms, throughput=207 IOPS
FAQ
Related Concepts
What's Next
You now understand I/O systems and device management. Next, learn about Linux namespaces and container isolation to understand how the OS virtualizes resources, or explore device drivers for writing kernel-level hardware code.
- Practice daily — Run
iostat -x 1and identify which Process is causing high I/O wait. - Build a project — Create a simple character device driver that implements a ring buffer with read/write operations.
- Explore related topics — Study io_uring for high-performance async I/O in Linux.
Built by the developers of Doda Browser, DodaZIP, and Durga Antivirus Pro.
Built by the developers of DodaTech
Doda Browser, DodaZIP & Durga Antivirus Pro