File System

What is it: A file system organizes raw disk blocks into files and directories. It provides the abstraction of named, hierarchically-organized data with permissions and timestamps.

Why: A raw disk is a flat array of 512-byte sectors. Without a file system, you track exact physical sectors yourself and manage free space manually. A file system provides: names (map names to data locations), free space management (know which blocks are available), metadata (size, permissions, timestamps), crash consistency (journaling — survive power loss without corruption).

graph LR d1["dir entry: 'file1.txt'"] --> inode1["inode #4092<br/>(size, perms, block pointers)"] d2["dir entry: 'file2.txt'"] --> inode1 inode1 --> blocks["data blocks"] note["Two names, one inode = a hard link.<br/>mv within same fs = just rewrites the dir entry."]

Inode: Every file has an inode — a fixed-size metadata structure containing: file size, owner UID/GID, permissions (rwx for owner/group/others), timestamps (atime/mtime/ctime), and pointers to data blocks (direct, indirect, double-indirect). The directory entry maps a filename string to an inode number. The inode does NOT contain the filename — that is in the directory. This is why mv file1 file2 in the same filesystem is instant (just changes the directory entry) while mv file1 /other-filesystem/file2 copies data.

Disk blocks: Data stored in fixed-size blocks (typically 4KB). A 1-byte file uses one full 4KB block — internal fragmentation. Files are not necessarily stored in contiguous blocks on disk. The file system tracks which blocks belong to which file via the inode's block list.

Why reading from disk is slower than RAM: RAM access: ~100ns. NVMe SSD: ~100µs (1000× slower). SATA SSD: ~500µs. HDD: seek + rotational latency + transfer = 5–20ms (100,000× slower). Additionally, disk reads go through: userspace read() → kernel VFS layer → block device driver → hardware → DMA to kernel buffer → copy to userspace. Multiple layers of indirection.

Sequential vs random access: Sequential read: data stored in consecutive blocks. OS read-ahead: kernel prefetches upcoming blocks while the application processes current ones. HDD: disk head moves linearly — maximum throughput. SSD: also benefits (fewer random-access overhead). Throughput: HDD ~100MB/s, NVMe SSD ~3000MB/s.

Random access: reads scattered across the disk. HDD: each read requires a disk seek (~5ms). 100 random reads = 500ms, regardless of data size. SSD: much better (no seek time), but still slower than sequential due to wear leveling overhead. This is why databases store related data on contiguous pages (B+Tree locality minimizes random I/O), and why append-only logs are faster than random updates.

Page cache (buffer cache): The kernel caches disk blocks in free RAM. A read() of a file first checks the page cache. On a hit, data returned from RAM (~100ns). On a miss, kernel issues a disk read, stores the result in the page cache, returns data. Subsequent reads of the same file are cache hits. This is why: first read of a file is slow, second read is fast. Rebooting flushes the page cache — first read after reboot is slow.

Journaling (crash consistency): Before modifying metadata (inode, directory entry, free space bitmap), the file system writes the intended changes to a journal (write-ahead log). If power fails mid-write, on recovery the journal is replayed. Without journaling: power failure mid-write leaves metadata in an inconsistent state — fsck must scan the entire disk to find and repair inconsistencies.

Trade-offs: File system overhead: directory lookup, inode access, and block mapping add ~100µs per file open. For workloads opening many small files (Node.js require()ing thousands of modules), this overhead is significant. Memory-mapped files (mmap) bypass some of this by mapping file content directly into the address space — reads go through the page cache without explicit read() syscalls.

Real-world usage: Proxel's ETL pipeline writes extracted data to temporary files on SSD, then bulk-loads into PostgreSQL. Sequential writes to temp files are fast. MinIO (S3-compatible) stores Xgist audio/video artifacts — object storage hides the file system behind an HTTP API but internally organizes data as sequential block writes.

Common pitfall

Deleting a file while another process still has it open doesn't actually free the disk space — Unix unlinks the directory entry immediately, but the inode (and its data blocks) stay allocated as long as any process holds an open file descriptor to it. A classic symptom: df shows a disk nearly full, but summing up all visible file sizes doesn't account for the gap — the missing space is held by deleted-but-still-open files, findable via lsof +L1 (or restarting the process holding them, which finally triggers the actual deallocation).