System Resource & I/O Optimization Questions

Direct answer

Read-ahead is the kernel (or an application) speculatively fetching data beyond what was explicitly requested, betting that a sequential access pattern will keep asking for the next chunk. It helps when access really is sequential and predictable, because it hides storage latency by having data ready before it's asked for. It hurts on random-access workloads, where it wastes I/O bandwidth and evicts genuinely useful cached pages fetching data that's never used. For a media streaming service serving large sequential reads, you'd deliberately increase the block-device readahead window well beyond the default.

Structured elaboration

OS-level read-ahead. The kernel detects a sequential access pattern to a file (successive reads at increasing offsets) and issues additional reads ahead of what the application asked for, populating the page cache before those blocks are requested. This is controlled per block device via the readahead setting, viewable and settable with blockdev --getra/--setra /dev/sdX (values are in 512-byte sectors), and persisted across reboots via a udev rule that reapplies the setting when the device is detected.

Application-level prefetching. Applications can give the kernel explicit hints rather than relying purely on heuristic detection: posix_fadvise(fd, ..., POSIX_FADV_SEQUENTIAL) tells the kernel to be more aggressive about read-ahead for this specific file descriptor, and POSIX_FADV_WILLNEED explicitly requests the kernel start prefetching a specific range immediately. Databases commonly do their own index-page prefetching above the OS layer, and media servers or download managers may issue explicit range-request prefetching over the network, a similar idea one layer up the stack.

When it helps. Any workload with a genuinely predictable, sequential access pattern: streaming a large media file, a full table scan, sequential log processing. The core mechanism is hiding storage latency by overlapping "fetch the next chunk" with "process the chunk you already have," so by the time the application asks for the next piece, it's already in cache.

When it hurts. Random-access workloads, most notably databases doing index lookups scattered across a large file, where the "next" blocks read-ahead fetches speculatively are usually never actually requested. This wastes I/O bandwidth on the storage device (competing with real requests) and pollutes the page cache by evicting genuinely hot pages to make room for speculative ones that go unused, hurting both throughput (wasted device bandwidth) and latency (real requests now queue behind speculative ones, and useful cache entries get evicted).

Tuning for a media streaming service. Increase the block device's readahead window well above the default (commonly 128 KB) to something sized for the streaming bitrate and access pattern, for example:

blockdev --setra 8192 /dev/nvme0n1

8192 sectors of 512 bytes each is 4 MB, a much larger anticipatory read window suited to sustaining large sequential streams at high bitrate with fewer, larger I/O operations rather than many small ones. Pairing this with an explicit posix_fadvise(fd, 0, 0, POSIX_FADV_SEQUENTIAL) call from the serving application lets the kernel be aggressive about read-ahead for that file from the very first read, rather than waiting for its own heuristics to detect the sequential pattern after a few reads have already happened. Because this trades page cache capacity for prefetch depth, validate against your actual cache hit ratio on a multi-tenant streaming box: a readahead window large enough to help one stream but big enough to evict other tenants' hot content is a net loss.

Worked example

Default readahead (blockdev --getra typically reporting 256 sectors, 128 KB) on a media server sustaining a 20 Mbps stream means the kernel issues many small anticipatory reads to keep up, each one a separate I/O operation with its own queueing and completion overhead. Increasing that to 8192 sectors (4 MB) means far fewer, larger I/O operations are needed to sustain the same stream, which reduces per-operation overhead and gives the kernel a much bigger buffer of already-fetched data to hide any transient storage latency behind. The trade-off becomes visible on a multi-tenant box: if 200 concurrent streams each get the configured 4 MB working set of read-ahead data cached (matching the 8192-sector/4 MB setting above), that's 200 x 4 MB = 0.8 GB of cache committed to prefetch alone, worth explicitly sizing against total available cache rather than assuming a larger number is free.

Trade-offs and pitfalls

Applying a large readahead window globally to a device that serves both sequential (streaming) and random (metadata lookups, small file serving) workloads is the most common mistake: the random-access traffic pays the read-ahead tax (wasted bandwidth, cache pollution) without getting any of the sequential-access benefit. If the same device serves mixed workloads, prefer per-file posix_fadvise hints from the application over a single global blockdev setting, so the tuning only applies to the traffic that actually benefits from it.

Direct answer

A CPU-bound workload spends most of its time actively computing, its speed is limited by processor cycles; an I/O-bound workload spends most of its time waiting on slow external operations (disk, network), its speed is limited by how fast those operations complete, not by processor speed.

Definitions

CPU-bound means adding faster or more CPU cores would meaningfully speed the workload up, because the CPU is the bottleneck: it is busy computing nearly all the time it is scheduled. Examples: video encoding, cryptographic hashing, in-memory data transformation. I/O-bound means the workload spends most of its wall-clock time blocked waiting for a disk read, a network response, or another slow external resource, with the CPU often sitting idle in between; adding CPU cores does little or nothing for it. Examples: a web server proxying requests to a slow downstream API, a batch job reading many small files from a spinning disk.

Detecting and distinguishing them in production

CPU usage (top, mpstat) sustained near 100% across cores, with %wa (iowait, the percentage of CPU time spent idle while waiting on outstanding disk I/O) low, points to CPU-bound. High %wa with CPU otherwise mostly idle points to I/O-bound: the CPU is not the constraint, it is waiting on something else. iostat -x 1 showing high %util (device busy percentage), elevated await (average I/O completion time), and disk throughput (rMB/s/wMB/s, megabytes read/written per second) that tracks the actual data volume moving, while application CPU usage stays low, confirms disk I/O as the bottleneck. For network-bound cases, high %wa will not show it (iowait specifically tracks disk I/O), so check connection counts, socket wait states, and application-level latency to a downstream dependency instead. vmstat 1 gives both signals side by side: the us/sy columns (user and system CPU time) versus the wa column in one view, useful for a quick first look.

Triage checklist: CPU-bound

Confirm with top/mpstat that CPU usage is genuinely saturated (not just one busy-looping thread on an otherwise idle multi-core box). Profile the hot code path (a sampling profiler, or language-level profiling tools) to find what is actually consuming cycles. Check for accidental inefficiency first: an algorithm with worse complexity than intended, a busy-wait loop instead of a blocking wait, or work being redundantly recomputed. Consider horizontal scaling (more instances) or algorithmic optimization; adding I/O capacity will not help here.

Triage checklist: I/O-bound

Confirm with iostat -x/vmstat that the CPU is genuinely mostly idle while wall-clock latency is high. Identify which resource: disk (iostat await/%util), network (connection/socket state, downstream service latency), or lock contention masquerading as I/O wait. Check for a fixable pattern: many small synchronous I/O calls that could be batched or made concurrent/asynchronous, a missing cache in front of a slow dependency, or a downstream service itself degraded. Consider caching, batching, asynchronous I/O, or a faster storage/network tier; adding CPU cores will not help here.

Trade-offs and pitfalls

The most common triage mistake is scaling the wrong resource: adding CPU to an I/O-bound service (or storage capacity to a CPU-bound one) burns money and time without fixing the actual bottleneck, so always confirm which regime you are in before reaching for a fix. A workload can also shift between the two under different load levels (I/O-bound at low concurrency because requests are mostly waiting, CPU-bound at high concurrency once enough concurrent work saturates the processor), so classify at the load level you actually care about, not just once in isolation.

Why swap is uniquely harmful for latency-sensitive services

A normal memory access completes in nanoseconds. When the touched page has been swapped out, that same access becomes a synchronous disk read, milliseconds on flash, tens of milliseconds on spinning disk: a five-to-six order of magnitude latency cliff. Worse, it is essentially random which page gets swapped and when it gets touched again, so it does not show up as a steady average-latency cost; it shows up as unpredictable p99/p999 tail-latency spikes (the slowest 1-in-100 or 1-in-1000 requests), which is exactly what latency SLAs (Service Level Agreements, the promised performance targets a service commits to) are built to protect and exactly what swap defeats. It is also invisible to the application: nothing in the code path 'knows' it just paid a disk-I/O cost for what looked like a memory read.

Techniques to avoid swap-induced latency

• Strict memory limits: set a cgroup memory.max hard cap comfortably under available RAM, and set memory.swap.max=0 (cgroups v2, the second-generation Linux resource-control hierarchy) so that cgroup can never swap at all; it gets OOM-killed instead. For a latency SLA, a fast, clean failure the orchestrator can restart is usually preferable to slow degradation nobody notices until it is too late.
• Overcommit tuning: vm.overcommit_memory=2 makes the kernel refuse to promise more memory than it can actually back (paired with vm.overcommit_ratio), preventing a host-wide memory crunch that would otherwise force swapping across every process on the box, not just the one that overcommitted.
• mlock: mlock()/mlockall() pin a process's critical working set in RAM so the kernel is not permitted to swap those pages out, at the cost of needing CAP_IPC_LOCK (the Linux capability that permits a process to lock memory) and a raised RLIMIT_MEMLOCK (the per-process limit on how much memory it may lock).
• cgroup settings: memory.low and memory.high are soft thresholds that trigger earlier, gentler reclaim before the hard memory.max is hit; disabling swap host-wide (swapoff -a) is a blunt but common choice on pure latency-sensitive fleets that accept OOM-kill as the failure mode.
• Memory reservation: capacity-plan so peak resident set size plus headroom fits comfortably in RAM; in Kubernetes terms, setting requests equal to limits for latency-critical pods avoids the scheduler bin-packing them too tightly.
• Admission control: shed or queue new work before memory pressure builds, using a memory high-watermark or PSI (Pressure Stall Information, a kernel metric for how much time processes spend blocked on a resource) signal as the trigger, so the service degrades on its own terms instead of the kernel's.
• Application-level fallbacks: circuit breakers that react to rising allocation/GC latency, backpressure on incoming requests, and exporting a memory-pressure health signal to the load balancer so an instance is pulled from rotation before it starts swapping, not after.

The common thread: prefer a controlled, fast failure (reject work, OOM-kill, fail over) to letting the kernel silently trade throughput and predictability for staying alive via swap.

Direct answer

The page cache is RAM the kernel uses to hold copies of disk blocks so repeat reads are served from memory instead of storage, and so writes can return quickly while the kernel flushes them to disk in the background. A write() call normally just copies data into the page cache and marks those pages dirty; it does not wait for the disk. fsync() is the call that forces those dirty pages (and, unlike fdatasync(), the file's metadata) to actually reach the storage device before it returns. Whether the page cache is helping or just eating memory comes down to one field: MemAvailable, not MemFree.

Structured elaboration

Buffered writes and write-back. By default, file I/O in Linux is buffered: write() copies your data into page-cache pages and returns immediately. Those pages are marked "dirty" (modified but not yet on disk). Kernel writeback threads (one per backing device) flush dirty pages to disk asynchronously, governed by two sysctls: vm.dirty_background_ratio (the percentage at which background flushing starts quietly) and vm.dirty_ratio (the percentage at which the writing process itself gets blocked until writeback catches up). Both percentages are taken against dirtyable memory, meaning free plus reclaimable pages, not against total RAM, so the byte ceiling they imply shrinks as anonymous memory grows; vm.dirty_bytes/vm.dirty_background_bytes set the same thresholds as absolute byte counts when you need a fixed bound. This is why a burst of writes can feel instant at first and then suddenly stall: you've crossed dirty_ratio and the kernel is now throttling you.

What fsync actually guarantees. fsync(fd) flushes all dirty pages belonging to that file descriptor to the underlying block device, and issues a cache-flush/FUA (force unit access) command so the write is committed past any volatile cache on the device itself, not just handed to the disk controller. Once fsync() returns successfully, that data survives a kernel crash or power loss, assuming the device honors flush commands correctly (a device with a broken or disabled write-cache flush can lie about this, which is a real failure mode to check for, not a theoretical one). fdatasync() gives the same data guarantee but skips flushing metadata that isn't needed to read the data back (e.g. mtime), which is cheaper when you don't need it.

Where O_DIRECT fits. O_DIRECT is an open flag that bypasses the page cache entirely for that file, transferring data close to directly between your buffer and the device. It exists for applications, mostly databases, that manage their own buffer pool and don't want the OS silently double-caching (and double-copying) the same pages. The trade-off: you give up the free page cache readahead and write coalescing the kernel would otherwise do, and you take on the responsibility for your own caching and I/O scheduling. O_DIRECT alone still does not guarantee durability: you generally still need fsync or O_DSYNC (an open flag that folds an fdatasync-equivalent durability guarantee into every write() call, syncing the data plus only the metadata needed to read it back; O_SYNC is the stronger sibling that folds in a full fsync-equivalent guarantee, including metadata like mtime that isn't needed to retrieve the data), because the storage device can have its own volatile write cache sitting below O_DIRECT.

Genuine benefit versus memory pressure. Don't judge this from free memory alone; idle RAM used for cache is not a problem by design. Instead:

• Check MemAvailable in /proc/meminfo (or the available column of free -h). This is the kernel's own estimate of how much memory could be freed for new allocations without swapping, after accounting for reclaimable cache. High MemAvailable relative to total RAM means the cache is healthy and reclaimable on demand.
• Check Dirty and Writeback in /proc/meminfo. If Dirty is climbing toward your dirty_ratio threshold, you're accumulating unflushed writes, not free performance.
• Watch vmstat 1 for si/so (swap in/out). Any sustained non-zero value means real pressure is forcing anonymous memory to disk, which is a different and much worse problem than "a lot of cache."
• Look at /proc/pressure/memory (pressure stall information, a kernel-reported metric of how much time processes actually stalled waiting on memory). Near-zero avg10 means processes aren't stalling; rising values mean the cache size is a symptom of real contention, not a free lunch.

Worked example

On a 64 GB box that's been serving reads for a while, free -h might show:

              total        used        free      shared  buff/cache   available
Mem:            62Gi        11Gi       1.2Gi       450Mi        49Gi        48Gi
Swap:            0B          0B          0B

49 GiB is sitting in buff/cache, but available is 48 GiB, almost the same. That tells you the kernel considers nearly all of that cache reclaimable right now: it's genuinely free capacity being used productively, not memory the system is straining to hold onto. Contrast that with a box where available has dropped to 4 GiB despite 49 GiB of buff/cache: that gap means most of that "cache" is actually dirty pages waiting on writeback, or the kernel is close to needing to reclaim aggressively, which is your early warning to check Dirty in /proc/meminfo and vmstat's si/so before it turns into swapping or an OOM (out-of-memory) kill, where the kernel terminates a process to free memory it can't reclaim any other way.

Trade-offs and pitfalls

The most common mistake is alerting on low MemFree directly, which pages nearly every healthy Linux box in existence, since the kernel is designed to use spare RAM for cache rather than leave it idle. Alert on MemAvailable instead. The second common mistake is assuming fsync guarantees durability against every failure mode: a storage controller with a volatile write cache that ignores flush commands, or a filesystem mounted with barriers disabled, can make fsync return success while data is still one power outage away from being lost. On database servers this is worth verifying explicitly rather than assuming the defaults are safe.

Approach

/proc/stat exposes cumulative kernel counters in units of jiffies (kernel clock ticks, usually 100 per second, i.e. USER_HZ=100) accumulated since boot. Each cpu/cpuN line has ten space-separated fields in a fixed order: user, nice, system, idle, iowait, irq (hardware interrupt handling time), softirq (deferred, software-handled interrupt time), steal, guest (CPU time spent running a virtual machine's code on behalf of this host), guest_nice (the same, for a niced/low-priority guest). Because these are running totals, a single snapshot tells you nothing about "current" utilization: you need two snapshots and the interval between them, then divide the delta of interest by the total delta across all ten fields for that window.

The core steps: parse each snapshot into {label: [ten integers]}, keep only labels present in both snapshots, compute per-field deltas with wraparound correction, sum the deltas for the total elapsed work, then each requested percentage is the matching field's delta divided by the total delta, times 100.

Wraparound handling: these are unsigned kernel counters (64-bit on modern kernels; some older/32-bit kernels expose them as 32-bit unsigned long). Subtracting a later value from an earlier one after the counter has wrapped past its maximum produces a negative raw delta. The fix is to add the modulus back whenever the raw delta is negative, which is standard unsigned-arithmetic correction, not a special case.

import json

FIELDS = ["user", "nice", "system", "idle", "iowait", "irq", "softirq", "steal", "guest", "guest_nice"]
WRAP_MOD = 2 ** 64  # modern kernels store these as unsigned 64-bit; use 2**32 if targeting a 32-bit kernel


def _parse_snapshot(text):
    rows = {}
    for line in text.strip().splitlines():
        parts = line.split()
        if not parts or not parts[0].startswith("cpu"):
            continue
        label = parts[0]
        values = [int(v) for v in parts[1:]]
        values += [0] * (len(FIELDS) - len(values))  # pad pre-guest-field kernels
        rows[label] = values[: len(FIELDS)]
    return rows


def _delta(before, after):
    d = after - before
    if d < 0:
        d += WRAP_MOD  # unsigned counter wrapped between snapshots
    return d


def cpu_utilization(snapshot1, snapshot2):
    before = _parse_snapshot(snapshot1)
    after = _parse_snapshot(snapshot2)
    result = {}
    for label in after:
        if label not in before:
            continue  # core only present in snapshot2 (hot-added): no valid delta yet
        deltas = [_delta(b, a) for b, a in zip(before[label], after[label])]
        total = sum(deltas)
        if total <= 0:
            result[label] = {k: 0.0 for k in ("user", "system", "idle", "iowait")}
            continue
        by_field = dict(zip(FIELDS, deltas))
        result[label] = {
            "user": round((by_field["user"] + by_field["nice"]) / total * 100, 2),
            "system": round((by_field["system"] + by_field["irq"] + by_field["softirq"]) / total * 100, 2),
            "idle": round(by_field["idle"] / total * 100, 2),
            "iowait": round(by_field["iowait"] / total * 100, 2),
        }
    return result

Key points

• This is a windowed measurement, not a point reading: both snapshots and the elapsed time between them matter.
• nice is folded into user, and irq/softirq into system, matching how most dashboards summarize CPU time into a small bucket set. That is a documented design choice here, not hidden behavior: keep them separate in by_field if a caller needs finer breakdown.
• iowait counts time a CPU was idle while at least one process was blocked on disk I/O. It is still technically idle CPU time (the core did no work), but it is broken out separately because a high iowait percentage points at storage latency, not compute demand, and the two need different fixes.
• The total divisor always sums all ten raw fields (including steal, guest, guest_nice), because the four reported percentages are shares of the full accounting window, not just of the four buckets being reported.

Complexity

Both parsing and the utilization computation are O(number of CPU lines), i.e. linear in core count, with O(1) work per field. No sorting or nested scans; memory use is a small fixed-size list per core.

Edge cases

• CPU hotplug: a cpuN label present in only one snapshot is skipped rather than crashing or reporting a bogus delta.
• Zero elapsed time (or a counter set that round-trips through a full wrap so every field's corrected delta happens to sum to zero): guarded to return zeros instead of dividing by zero.
• Pre-guest/guest_nice kernels (eight-field lines): padded to the fixed ten-field width.
• Counter wraparound: corrected via the modulus, shown in the worked example below.
• Malformed input is intentionally not swallowed here (int() will raise on a corrupt line): in a production collector you would wrap the parse in a try/except per line so one bad read does not take down the whole scrape.

Worked example

Two snapshots, roughly one second apart, two cores:

snapshot1:
cpu  3357 0 4313 1362393 234 0 0 0 0 0
cpu0 1679 0 2156 681000 117 0 0 0 0 0
cpu1 1678 0 2157 681393 117 0 0 0 0 0

snapshot2:
cpu  3427 0 4358 1362473 334 0 0 0 0 0
cpu0 1739 0 2186 681010 217 0 0 0 0 0
cpu1 1688 0 2172 681463 117 0 0 0 0 0

By hand for the aggregate cpu line: user delta 70, system delta 45, idle delta 80, iowait delta 100, total 295. user% = 70/295100 = 23.73, system% = 45/295100 = 15.25, idle% = 80/295100 = 27.12, iowait% = 100/295100 = 33.90. Running cpu_utilization(snapshot1, snapshot2) and printing the result confirms this exactly:

{
  "cpu":  {"user": 23.73, "system": 15.25, "idle": 27.12, "iowait": 33.9},
  "cpu0": {"user": 30.0,  "system": 15.0,  "idle": 5.0,   "iowait": 50.0},
  "cpu1": {"user": 10.53, "system": 15.79, "idle": 73.68, "iowait": 0.0}
}

cpu0 is doing a real burst of both compute and disk-wait work (50% iowait, only 5% idle) while cpu1 is mostly idle, which the aggregate cpu line alone would have hidden by averaging the two cores together.

For the wraparound path, feeding a synthetic pair where the counter has rolled past its modulus (before=18446744073709500000, after=200) still yields a sane, bounded percentage (100% user, since only that field moved) instead of the huge negative-delta garbage a naive subtraction would produce.

Trade-offs and pitfalls

• The sampling interval is a smoothing choice: a sub-second window is noisy and amplifies short bursts; production agents commonly sample every 5 to 15 seconds and rely on a dashboard's own rate/window function for further smoothing.
• Reading only the aggregate cpu line hides per-core imbalance (a single hot core in an otherwise idle box), which matters for diagnosing single-threaded bottlenecks; always keep the per-cpuN breakdown available for that kind of alerting.
• Do not confuse this with per-process accounting: /proc/stat is whole-system; a specific process's CPU time lives in the utime/stime fields of /proc/<pid>/stat, which is a different counter space entirely.
• On a virtualized host, a nonzero and growing steal share (time the hypervisor gave to other tenants instead of this VM) is itself an important capacity signal even though the question's four requested buckets do not surface it directly; it still counts toward the total divisor here, which is correct, but a monitoring agent should usually report it too.