File Systems¶

What a file system does¶

A storage device is a flat array of blocks numbered 0 to N. A file system turns it into named files in a directory tree, and must answer, for every operation:

• Naming: which file does /data/db/000123.sst refer to?
• Allocation: which blocks hold that file's bytes, and which blocks are free?
• Metadata: size, owner, permissions, timestamps.
• Crash consistency: after a power loss in the middle of an update, is the structure still valid?

A storage engine depends on all four, and its durability guarantees are only as good as its understanding of the last one.

On-disk structures (ext4 as the example)¶

Inodes¶

Every file and directory has an inode: a fixed-size record holding its metadata and the location of its data — but not its name.

ls -i shows inode numbers; stat file shows the whole inode.

Directories are files of names¶

A directory's content is a list of (name → inode number) entries. Resolving /data/db/000123.sst reads the root directory's entries to find data's inode, reads that directory to find db, then db to find 000123.sst's inode — each step possibly a disk read, though the kernel's dentry cache keeps recent lookups in memory. Large directories are indexed (ext4 htree, XFS B+Trees) so lookup in a directory of a million files doesn't scan them all.

Because names live in directories:

• A hard link is just another directory entry with the same inode number; the link count tracks how many. Data is freed only when the link count is 0 and no process has the file open (see File Descriptors).
• rename within one file system only edits directory entries — it is instant regardless of file size, and POSIX guarantees it is atomic: the target name refers to the old file or the new one, never neither. Moving across file systems is a copy plus delete.
• RocksDB's checkpoints and backups hard-link immutable SSTables into a new directory: a full point-in-time copy of a large database in milliseconds, using no extra space.

Extents and allocation¶

Old file systems stored a pointer per 4 KB block (with indirect blocks for large files). Modern ones store extents: (logical block, physical block, length) runs — one extent can describe 128 MB of contiguous file. The allocator tries hard to keep files contiguous:

• Delayed allocation: blocks are chosen at writeback time, not at write() time, when the final size is better known.
• Preallocation: fallocate(fd, 0, 0, size) reserves contiguous blocks up front. A storage engine preallocating its WAL avoids both fragmentation and metadata updates on every append — making fdatasync cheaper, since the file size no longer changes.

Block size and small files¶

Space is allocated in blocks (typically 4 KB). A 100-byte file occupies a full block, so millions of tiny files waste space and inodes (ext4 allocates a fixed number of inodes at format time; running out gives ENOSPC with free space remaining — check with df -i). This is one reason databases put many records into large files rather than one file per record.

The VFS layer¶

Linux supports dozens of file systems behind one interface. The VFS (virtual file system) defines generic objects — superblock, inode, dentry, file — and the operations on them; each file system implements them. System calls (open, read, rename) go to the VFS, which dispatches to ext4, XFS, Btrfs, NFS, tmpfs, or procfs. That is why /proc/<pid>/maps can be read with cat even though no disk is involved.

Crash consistency: the core problem¶

A single logical operation touches several on-disk structures. Appending a block to a file:

1. allocate a block (update the free-space bitmap),
2. write the data into the block,
3. update the inode (size, extent).

A power loss can leave any subset persisted, in any order the disk chose:

fsck: the old answer¶

After an unclean shutdown, fsck scanned every inode, directory, and bitmap to find and repair inconsistencies. On a multi-terabyte disk that took hours.

Journaling¶

Journaling file systems (ext3/4, XFS, NTFS) apply the write-ahead log idea to metadata:

1. Write the intended changes (the new inode, bitmap, directory blocks) to a journal area, followed by a commit record.
2. Only after the commit is durable, write them to their real locations (checkpoint).
3. After a crash, replay committed transactions from the journal; discard uncommitted ones. Recovery takes seconds.

What is journaled matters. ext4's modes:

Journaling keeps the file system consistent. It does not make your multi-step updates atomic, and it does not make write() durable. A database still needs its own WAL, checksums, fsync, and the rename/directory-sync protocols in Durability and fsync.

Copy-on-write file systems¶

Btrfs, ZFS, and APFS never overwrite blocks in place; they write new blocks and atomically switch a root pointer, like a copy-on-write B+Tree. They get crash consistency without a journal, cheap snapshots, and end-to-end checksums of data — at the cost of fragmentation for in-place-update workloads such as databases. See Copy-on-Write.

Log-structured file systems¶

LFS designs (and F2FS for flash) write everything — data and metadata — sequentially into a log and garbage-collect old segments: the same design as an LSM tree, with the same cleaning (compaction) costs.

Which file system for a database¶

Useful commands¶

stat 000123.sst                  # inode, size, blocks, timestamps, links
filefrag -v 000123.sst           # extents: how fragmented is the file
df -h ; df -i                    # space and inode usage
lsof +L1                         # deleted files still held open
findmnt -o TARGET,FSTYPE,OPTIONS # mount options (data=ordered, noatime, ...)

noatime (or the default relatime) avoids turning every read into a metadata write to update the access time.

Real-world usage: Proxel's ETL pipeline writes extracted data to temporary files on SSD, then bulk-loads into PostgreSQL; sequential writes to temp files are fast. MinIO (S3-compatible) stores Xgist audio/video artifacts — object storage hides the file system behind an HTTP API but still lays data out as large sequential files internally.

Common pitfall¶

Deleting a file while another process still has it open doesn't free the disk space — the directory entry disappears immediately, but the inode and its blocks stay allocated as long as any open file description references it. The classic symptom: df shows a disk nearly full, but summing the visible files doesn't account for the gap. The missing space belongs to deleted-but-open files (typically logs removed by a rotation script while the service keeps writing), found with lsof +L1 and released only when the holding process closes them or restarts. Log rotation should signal the process to reopen its log file, or copy-and-truncate, rather than delete a file that is in use.