System Resource & I/O Optimization Questions

Approach

Stream in fixed-size chunks straight from the request body to the destination (disk or object storage) instead of buffering the whole file, checksum incrementally as bytes flow through rather than in a separate pass, track how many bytes have landed so an interrupted transfer can resume from that offset instead of restarting, and let a bounded concurrency limit (not an unbounded goroutine or thread per upload) be the backpressure mechanism under load.

Design

Never load the whole file into memory: read a fixed-size chunk (a few megabytes is typical), write it through immediately, and discard it before reading the next chunk, so peak memory use is bounded by the chunk size, not the file size, regardless of whether the file is 10MB or 10GB. Chunked or multipart upload straight to disk or object storage: for object storage specifically, most providers expose a multipart upload API (initiate, upload part N with its own ETag, complete) that maps naturally onto this same fixed-chunk read loop, each chunk becomes one part. Resumability: persist the highest successfully-written offset (or the list of successfully completed part numbers for multipart object storage) somewhere durable (a small metadata record, not just in-process memory), so a retry after a dropped connection can query that offset and resume from there instead of re-uploading bytes already safely stored. Backpressure: the destination write call itself provides natural backpressure for a single stream, if the write blocks because the disk or object store is slow, the next read from the source simply waits, so the service does not keep pulling data faster than it can be persisted; for many concurrent uploads, a bounded worker pool (a fixed-size semaphore or channel) caps how many uploads are in flight at once, so total memory use stays bounded even under a burst of concurrent requests. Integrity verification: compute a checksum (SHA-256 or a similar strong hash) incrementally as each chunk streams through, feeding the same bytes into both the destination writer and the hash function in one pass, rather than re-reading the whole file afterward to verify it, which would defeat the memory-bounded design.

Worked example (Go)

const chunkSize = 4 << 20 // 4 MiB per chunk: never buffer the whole object

// streamChunks reads src in chunkSize pieces and writes each straight
// through to dst while it reads, returning a hex sha256 of the whole
// stream. Because writes happen inline with reads, a slow or blocked
// destination stalls the next Read call automatically: that stall IS
// the backpressure, no separate signalling channel needed for one stream.
func streamChunks(dst io.Writer, src io.Reader) (string, int64, error) {
    h := sha256.New()
    w := io.MultiWriter(dst, h)
    buf := make([]byte, chunkSize)
    var total int64
    for {
        n, rerr := src.Read(buf)
        if n > 0 {
            written, werr := w.Write(buf[:n])
            total += int64(written)
            if werr != nil {
                return "", total, fmt.Errorf("write at offset %d: %w", total, werr)
            }
        }
        if rerr == io.EOF {
            break
        }
        if rerr != nil {
            return "", total, fmt.Errorf("read at offset %d: %w", total, rerr)
        }
    }
    return hex.EncodeToString(h.Sum(nil)), total, nil
}

// uploadConcurrent fans independent files out to a bounded worker pool.
// maxInFlight is the concurrency-level backpressure knob for the
// multi-upload case: unbounded goroutines-per-upload wins a
// microbenchmark and then exhausts memory or file descriptors under
// real concurrent load.
func uploadConcurrent(paths []string, destDir string, maxInFlight int) map[string]string {
    sem := make(chan struct{}, maxInFlight)
    var wg sync.WaitGroup
    var mu sync.Mutex
    results := make(map[string]string)
    for _, p := range paths {
        wg.Add(1)
        go func(p string) {
            defer wg.Done()
            sem <- struct{}{}        // blocks once maxInFlight are in flight
            defer func() { <-sem }()
            src, err := os.Open(p)
            if err != nil {
                return
            }
            defer src.Close()
            dst, err := os.Create(filepath.Join(destDir, filepath.Base(p)))
            if err != nil {
                return
            }
            bw := bufio.NewWriterSize(dst, chunkSize)
            sum, _, err := streamChunks(bw, src)
            if err == nil {
                bw.Flush()
                mu.Lock()
                results[p] = sum
                mu.Unlock()
            }
            dst.Close()
        }(p)
    }
    wg.Wait()
    return results
}

Run against six ~9MB source files with maxInFlight=3, each result's SHA-256 checksum was independently recomputed by reading the destination file back and compared to what streamChunks produced inline:

file0.bin: checksum_match=true
file1.bin: checksum_match=true
file2.bin: checksum_match=true
file3.bin: checksum_match=true
file4.bin: checksum_match=true
file5.bin: checksum_match=true
all_checksums_match: true

What changes for a Go service maximizing throughput under concurrent load

maxInFlight becomes the primary tuning knob: too low under-utilizes available disk/network bandwidth, too high causes goroutines to pile up waiting on a slow destination and inflates memory (each blocked goroutine still holds its chunk buffer); size it from measured throughput per upload and the destination's actual concurrent-write capacity, not a guess. Reuse chunk buffers across uploads with sync.Pool rather than allocating a fresh chunkSize buffer per upload, since allocation and garbage collection of large buffers under high concurrency becomes a real throughput cost at scale (garbage collection, the Go runtime's automatic memory reclamation, has more work to do the more short-lived large objects it must track). For multipart uploads to object storage specifically, upload multiple parts of the SAME file concurrently (not just multiple files concurrently), since a single large file's throughput is otherwise capped by one sequential part-upload stream even while overall service concurrency looks high. Use io.CopyBuffer or the pattern above rather than ioutil.ReadAll anywhere in the path, since the latter reintroduces full in-memory buffering and defeats the design's whole point.

Trade-offs and pitfalls

Incremental checksumming means the destination receives unverified bytes as they stream; if integrity must be confirmed before the object is considered durable, either use a storage API that verifies a provided checksum against what it received (many object stores support this natively as part of the multipart-complete call) or accept a brief window where a corrupted upload is only caught after the fact and must be re-uploaded. Persisting resume state adds its own reliability requirement: if the resume-offset record itself is lost or inconsistent with what actually landed at the destination, resuming from a stale offset can either skip bytes or duplicate them, so treat the offset record with the same durability care as the data itself. Bounding concurrency for throughput requires real measurement (vary maxInFlight and measure actual aggregate throughput and destination-side error rate) since the optimal value depends on the destination's own concurrency limits, not just the source service's own capacity.

Approach

/proc/meminfo is a Linux virtual file (generated by the kernel on read, not stored on disk) with one Key: value kB pair per line; parse it into a dictionary keyed by field name rather than by fixed line position, since the kernel does not guarantee field order is stable across kernel versions, then derive used memory and percent used from the four fields the question names, treating any missing field as an explicit None rather than crashing or silently defaulting to zero.

Implementation (Python)

def parse_meminfo(text):
    """Parse /proc/meminfo text into a dict with derived used-memory fields.

    Handles missing fields (returns None for anything not present) and
    does not assume any particular line order, since /proc/meminfo's
    field order is not a guaranteed kernel ABI.
    """
    wanted = {"MemTotal", "MemFree", "Buffers", "Cached", "SwapTotal", "SwapFree"}
    values = {}
    for line in text.splitlines():
        if ":" not in line:
            continue
        key, _, rest = line.partition(":")
        key = key.strip()
        if key not in wanted:
            continue
        # Values look like "   16345600 kB"; keep the number, kB is the base unit.
        parts = rest.strip().split()
        if not parts:
            continue
        try:
            values[key] = int(parts[0])
        except ValueError:
            continue

    result = {k: values.get(k) for k in wanted}

    mem_total = result["MemTotal"]
    mem_free = result["MemFree"]
    buffers = result["Buffers"]
    cached = result["Cached"]

    if None in (mem_total, mem_free, buffers, cached):
        result["used_kb"] = None
        result["percent_used"] = None
    else:
        used = mem_total - mem_free - buffers - cached
        result["used_kb"] = used
        result["percent_used"] = round((used / mem_total) * 100, 2) if mem_total else None

    return result

Output

Ran against a normal sample (all six fields present, in a different order than the code checks them in) and a partial sample (SwapTotal/SwapFree lines entirely absent, which some kernels omit when swap is disabled):

{
  "Cached": 4271208,
  "MemFree": 6531220,
  "SwapFree": 2097148,
  "SwapTotal": 2097148,
  "Buffers": 412872,
  "MemTotal": 16336984,
  "used_kb": 5121684,
  "percent_used": 31.35
}
{
  "Cached": 4271208,
  "MemFree": 6531220,
  "SwapFree": null,
  "SwapTotal": null,
  "Buffers": 412872,
  "MemTotal": 16336984,
  "used_kb": 5121684,
  "percent_used": 31.35
}

Hand-checked: 16336984 - 6531220 - 412872 - 4271208 = 5121684, and 5121684 / 16336984 * 100 = 31.354..., which rounds to 31.35, matching the printed output; the partial-sample case correctly still computes used_kb/percent_used (they only depend on the four non-swap fields) while reporting SwapTotal/SwapFree as null since those lines were genuinely absent from the input.

Key points

Order-independence: the parser reads every line and keys results by field name (key.strip() from before the colon), so it does not matter whether MemTotal is line 1 or line 12, unlike a parser that assumes a fixed line index. Missing fields: each wanted field defaults to None via values.get(key) rather than 0, which matters because a missing SwapTotal (swap disabled) is a meaningfully different fact than SwapTotal: 0 (swap enabled with zero configured, unusual but different), and silently treating "absent" as "zero" would hide that distinction from anyone consuming the JSON. Derived fields only compute when all four required inputs are present; the arithmetic itself (MemTotal - MemFree - Buffers - Cached) uses the exact formula the question specifies, and both derived fields fall back to None together rather than partially computing on incomplete data.

Trade-offs and pitfalls

This definition of "used" memory is the classic (and slightly pessimistic) one that treats Buffers and Cached as fully reclaimable, which is close to true but not exact on modern kernels (some cached memory is not trivially reclaimable, and there is a more accurate MemAvailable field the kernel exposes directly for this purpose since Linux 3.14). For a production monitoring script, prefer reading MemAvailable directly over recomputing this classic formula, since the kernel's own estimate accounts for reclaim nuances this simple subtraction does not; the formula in this answer follows the question's explicit specification. All units here stay in kB (the native unit /proc/meminfo reports) throughout, converting to MB or GB and mixing units partway through the calculation is a common source of off-by-1024x bugs in scripts like this.

Direct answer

No, low free memory with high cached memory does not by itself mean the system is under memory pressure, and most healthy Linux servers look exactly like this. Linux deliberately spends idle RAM on the page cache (recently read or written disk blocks kept in memory) because unused RAM produces zero benefit, and that cache is normally reclaimed instantly when something else needs the memory. The command that actually tells you whether this is dangerous is checking MemAvailable in /proc/meminfo, not MemFree.

Structured elaboration

MemFree only counts memory that is doing absolutely nothing. Cached counts page-cache pages, most of which are "clean" (matching what's on disk, so they can be dropped for free at any moment) but some of which may be "dirty" (modified but not yet written back, which can't be dropped until they're flushed). MemAvailable is a kernel-computed estimate that starts from MemFree and adds back the portion of Cached, buffers, and reclaimable slab memory (kernel-internal caches such as inode and dentry caches) that could actually be freed for a new allocation without triggering swapping. It exists specifically because MemFree alone is a misleading health signal on any system that's been running for more than a few minutes.

Commands to validate:

• free -h: the available column is MemAvailable in human-readable form. Compare it against total.
• cat /proc/meminfo | egrep 'MemFree|MemAvailable|Cached|Dirty|Writeback': gives you the raw numbers plus the dirty/writeback state, which tells you how much of the cache is unflushed writes rather than reusable read cache.
• vmstat 1 5: watch the si and so columns (swap in/out, in KB per second). Any sustained non-zero value means the kernel is actually paging anonymous memory to disk right now, which is the real evidence of pressure, not the cache size.
• cat /proc/pressure/memory: pressure stall information (PSI), a kernel metric reporting the percentage of time processes were actually stalled waiting for memory. avg10 near 0 for both the some and full lines means nothing is waiting.
• dmesg -T | grep -i 'killed process': confirms whether the out-of-memory (OOM) killer, the kernel mechanism that terminates a process to free memory when reclaim can't keep up, has actually fired recently.

Worked example

Suppose /proc/meminfo on a 32 GB host shows:

MemTotal:       32937984 kB
MemFree:          612340 kB
MemAvailable:   29841200 kB
Cached:         28104512 kB
Dirty:              8420 kB
Writeback:             0 kB

MemFree is only about 600 MiB, which looks alarming in isolation. But MemAvailable is about 28.5 GiB of the 31.4 GiB total, meaning the kernel considers about 91% (29841200 / 32937984) of the box's memory reclaimable on demand. Mind the unit while you do that conversion: /proc/meminfo prints these values in KiB even though it labels the column kB, so 32937984 is 31.4 GiB (a 32 GiB host), not the "32.9 GB" you get by dividing by a million. The percentage is unaffected either way, because both sides of the ratio carry the same unit, which is exactly why the ratio is the number to trust and the human-readable conversion is the one to be careful with. Dirty is a tiny 8 MiB, nowhere near a dirty_ratio threshold, and Writeback is 0, so there's no backlog of unflushed writes. Paired with vmstat showing si/so at 0, this is a textbook healthy caching pattern: don't act on it.

Now compare a second host where MemAvailable has fallen to 1.2 GiB out of the same 31.4 GiB, Dirty is climbing past several hundred MB, and vmstat shows non-zero so (pages being swapped out). That's the dangerous pattern: available memory is genuinely thin, some of the "cache" is unflushed writes rather than reusable read cache, and the kernel has already started swapping to cope, which will show up as latency spikes on anything that touches those swapped-out pages next.

Trade-offs and pitfalls

The most common on-call mistake is paging someone because free -h's free column looks small; that column is close to meaningless on a running Linux box. A subtler pitfall is trusting Cached as fully reclaimable without also checking Dirty/Writeback: a system doing heavy buffered writes can show a large cache number that is mostly unflushed data, not free capacity, which behaves very differently under pressure than clean, reusable read cache.

Direct answer

A context switch is the kernel saving one process or thread's CPU state and loading another's so the CPU can run something else; voluntary switches happen when a task gives up the CPU itself (blocking on I/O or a lock), involuntary switches happen when the kernel scheduler preempts a still-runnable task to give another task a turn. Excessive switching adds latency because every switch has real overhead (saving/restoring registers, flushing CPU caches) that is pure cost, not useful work.

Voluntary versus involuntary

A voluntary context switch happens when a running task calls something that blocks it: waiting on a disk read, a network socket, a mutex/lock held by another thread, or an explicit sleep. The task chooses to yield the CPU because it has nothing useful to do until that operation completes. An involuntary context switch happens when a task is still runnable and has more work to do, but the kernel scheduler preempts it anyway, typically because its time slice expired or a higher-priority task became ready to run. The distinction matters diagnostically: a high voluntary rate usually points to an I/O- or lock-heavy workload; a high involuntary rate usually points to CPU contention, too many runnable tasks competing for too few cores.

How excessive switching affects latency

Each context switch costs real, non-trivial CPU time directly (saving and restoring the CPU's register state) and indirectly (the new task's working set is not in the CPU's L1/L2 cache, so its first several memory accesses are slower cache misses until the cache "warms up" again for that task, an effect called cache pollution). A workload doing many small units of work interleaved with frequent blocking or preemption pays this overhead repeatedly, which shows up as added tail latency even though each individual switch only costs microseconds, because the cost compounds across thousands of switches per second on a busy host.

Metrics that show context-switch rates

vmstat 1 reports cs (context switches per second) directly in its system column group, alongside in (interrupts per second) for comparison; a sustained cs in the tens of thousands per second on a modest host is worth investigating. /proc/stat exposes a cumulative ctxt counter (total context switches since boot), useful for computing a rate over a custom interval by sampling it twice and dividing the delta by the elapsed time. Per-process detail is available via /proc/<pid>/status, which reports voluntary_ctxt_switches and nonvoluntary_ctxt_switches counters separately, letting you attribute excessive switching to a specific process rather than only seeing a host-wide aggregate.

Strategies to reduce excessive context switching

At the kernel level: pin latency-sensitive processes to dedicated CPU cores with taskset/isolcpus so the scheduler is not constantly moving them or preempting them for unrelated work; tune scheduler parameters (sched_min_granularity_ns and related sysctl knobs) to give tasks longer time slices, trading some fairness for less switching overhead; use a real-time scheduling class (SCHED_FIFO/SCHED_RR) for the small number of processes that genuinely need to avoid preemption. At the application level: replace thread-per-request models with fewer threads and asynchronous, event-driven I/O (an event loop blocks and switches far less often than hundreds of threads each individually blocking on their own I/O); reduce lock contention (shorter critical sections, finer-grained locks, lock-free data structures where appropriate) since every lock a thread blocks on is a voluntary context switch; batch small operations together so the process yields the CPU less often for the same amount of useful work.

Trade-offs and pitfalls

A high cs rate is not automatically bad: a healthy, well-utilized multi-tenant host with many short-lived I/O-bound tasks will naturally show a high voluntary switch rate, and that is often just the workload being efficient about not blocking the CPU while waiting. The signal worth chasing is a high involuntary rate combined with rising tail latency, which points to genuine CPU contention rather than healthy I/O interleaving; conflating the two leads to tuning the wrong thing.

Direct answer

Use the cgroup v2 io controller (the cgroup v1 equivalent was called blkio; both throttle block-layer I/O per cgroup) to either give each container a relative weight (fair-share) or a hard cap on IOPS (input/output operations per second) and throughput (io.max). Weights are fairer under contention and waste less capacity when a container is idle; hard caps give predictable, guaranteed ceilings but can leave a device under-utilized when only one tenant is active.

Discussion

Implementation: io.weight (default 100, range 1-10000) sets a relative share of disk time per device, visible per cgroup at /sys/fs/cgroup/<cgroup>/io.weight. io.max sets hard per-device caps in four independent dimensions: rbps/wbps (read/write bytes per second) and riops/wiops (read/write operations per second), e.g. echo "8:0 rbps=50000000 wbps=20000000" > io.max caps device major:minor 8:0 to 50MB/s reads and 20MB/s writes. Container runtimes (Docker, containerd, Kubernetes via the kubelet) expose both as --blkio-weight / io.weight and per-device IOPS/bandwidth limits in the container spec, which map straight onto these cgroup files.

Fairness: weights only redistribute contention, they do not reserve anything; a cgroup with weight 500 gets roughly 5x the disk-time share of one at weight 100, but only when both are actively issuing I/O. On an idle device a low-weight cgroup can still use 100% of the available throughput, which is usually desirable (unused capacity should not go to waste) but means weight alone does not guarantee a worst-case bound.

Burst behavior: io.max has no built-in token-bucket burst allowance the way, say, a network traffic shaper does; there's no mechanism to bank unused I/O capacity while idle and spend it later. What looks like burst tolerance is really an artifact of enforcement granularity: the block-layer throttle evaluates the configured rbps/riops/wbps/wiops caps over a short rolling accounting window rather than instantaneously, so a workload can briefly exceed the nominal cap within that window before the throttle catches up and starts delaying requests. io.latency is a separate, unrelated cgroup v2 controller and does not give the cgroup it's applied to any burst allowance for io.max; it works the other direction, protecting one cgroup's I/O latency by throttling other, lower-priority cgroups on the same device once the protected cgroup's measured latency exceeds its configured target. If you actually need deliberate burst-then-throttle behavior (accumulate credit while idle, spend it under load, similar to cloud burstable-volume IOPS credits), the kernel's io controller doesn't provide that natively for io.max; it has to be built above the kernel, in the application or a sidecar. Weight-based fairness (io.weight) has no burst concept either; it is purely proportional at whatever instant contention exists.

Latency impact: hard IOPS/bandwidth caps convert what would be a fast I/O completion into an artificially delayed one once the cap is hit, which shows up as increased read/write latency inside the throttled container even though the underlying device has spare capacity, because the kernel is intentionally queuing or rejecting requests to enforce the cap. This is the most common surprise: a team sees their own container's latency spike after ops adds an I/O limit and assumes the disk is failing, when actually the limit is working as designed.

Verification: read back the cgroup's own I/O accounting at /sys/fs/cgroup/<cgroup>/io.stat, which reports rbytes, wbytes, rios, wios per device and lets you confirm actual throughput matches the configured cap under load; cross-check with iostat -x 1 on the host to see aggregate device utilization, and with the application's own latency metrics to see the throttling's user-visible cost. Load-test with fio (Flexible I/O Tester, a standard Linux disk I/O benchmarking tool) targeting the container specifically (not the host) to validate the cap holds under the container's real I/O pattern (random 4K vs sequential 1M writes throttle very differently even at the same nominal cap).

Limitations: the io controller only throttles requests that go through the block layer with a cgroup context attached, so buffered writes that hit the page cache first and get written back later by a kernel writeback thread can, on older kernels, lose their cgroup association and evade the cap (this is largely fixed on modern kernels with cgroup-aware writeback, but worth verifying on the specific kernel version in use). Writes issued with O_DIRECT (an open flag that bypasses the page cache so I/O goes straight from the application's buffer to the block layer) sidestep that specific writeback-attribution problem, since there is no intermediate writeback thread to lose the association, but O_DIRECT I/O is just as subject to io.max/io.weight caps as any other block-layer request; it does not grant an exemption. Caps are per-device, so a container spanning multiple physical devices or a logical volume striped across several disks needs the limit applied at the right layer to actually constrain aggregate I/O. Network-attached storage (NFS, iSCSI) is not always cleanly covered by the block-layer io controller in the same way local block devices are.

Trade-offs and pitfalls

Weights are the right default for fair-sharing a busy shared disk; hard caps (io.max) are the right tool when you need a guaranteed, predictable ceiling for capacity planning or SLA (service-level agreement, a commitment about the level of service provided) purposes, at the cost of wasting device capacity when the capped container is the only active tenant. The single biggest pitfall is setting a cap without first measuring the container's actual I/O pattern with a representative benchmark; a cap that looks generous for sequential throughput can be crushing for a workload that is actually IOPS-bound on small random reads, since bytes/sec and ops/sec are independent dimensions.