Performance Profiling & Bottleneck Analysis Questions

Framing

RSS (Resident Set Size, the actual physical memory a process is using) growing 100MB every hour toward an OOM kill has three plausible explanations that need different fixes: a native memory leak, a GC (garbage collection) tuning/timing issue, or intentional cache growth. The job is to tell them apart before touching anything.

Step 1: Confirm the trend is real

Sample RSS at a consistent interval over several hours (not one-off snapshots) to confirm steady growth rather than normal fluctuation.

Step 2: On-heap or off-heap?

If the process runs on a managed runtime (JVM, Node.js, Python, Go), compare the runtime's own reported heap usage (e.g. JVM heap used via jstat, Node's process.memoryUsage().heapUsed) against total RSS. If RSS is climbing much faster than the managed heap, the growth is happening OUTSIDE the runtime's tracked heap, e.g. a native extension, off-heap buffers, or direct malloc calls the runtime doesn't account for.

Step 3a: If it's on-heap, leak vs GC timing vs intentional cache

Force a full collection where safe (e.g. jcmd <pid> GC.run) and watch whether heap-used snaps back down. If it does, the "growth" was just reclaimable garbage waiting on a collection cycle, not a leak. If a cache is by design, check whether it has already hit its configured max size; if RSS keeps climbing past that configured ceiling, the cache's own bound isn't being enforced, that's a bug in the cache, not expected behavior.

Step 3b: If it's off-heap, find the allocator

Use OS-level tools: pmap <pid> to see which memory regions (anonymous heap, mmap'd regions) are actually growing, or a native allocator's own leak-detection mode (jemalloc/tcmalloc), or a heap profiler like valgrind --tool=massif for a detailed allocation profile, usually reserved for staging given its overhead.

Step 4: Correlate with load

If RSS keeps growing even at near-zero traffic, that's a strong leak signal, nothing legitimate should grow while idle. If growth is roughly proportional to request volume, that points more toward a cache or a per-request accumulation that isn't being cleaned up.

Short-term mitigations

• Set or lower an explicit max-size and eviction policy if it's a cache.
• Set a memory limit via a cgroup (control group, a Linux kernel mechanism for capping a process's resource usage) with alerting well before the OOM threshold, instead of running to the kill.
• Schedule a rolling restart tied to a memory threshold as a stopgap while root-causing.
• If a recent deploy lines up with when growth started, consider rolling it back as the fastest mitigation.

Isolate the flags from the code diff

First rule out that the regression is actually a code change that landed alongside the flag change. Take the exact same source commit and build it twice, once with the old flag set and once with the new one, and re-run the same benchmark. If the regression only appears with the new flags on identical source, it is the compiler, not the code; if it appears either way, you are chasing the wrong cause.

Bisect the flag set, not just commits

A "compiler optimization flags" change is usually several flags landing together (inlining budget, bounds-check behavior, a new codegen pass). Treat the flag set like a commit range and binary-search it: build with half the new flags, benchmark, repeat until one flag is isolated. A single toggle can explain most of a fleet-wide regression even when only one hot function is affected. For example, disabling inlining alone on a small hot loop:

BenchmarkSumAbs (inlining on,  default)     -14   243 ns/op   0 B/op
BenchmarkSumAbs (inlining off, -gcflags=-l) -14   889 ns/op   0 B/op

That single flag alone produced a 3.7x regression on this function; a fleet-wide 10% aggregate regression is entirely consistent with one flag hitting a handful of hot functions this hard while leaving everything else unaffected.

Use unstripped builds while investigating

A "stripped" build (for Go, built with -ldflags="-s -w") removes the symbol table and debug info to shrink the binary. That is fine for a production artifact but useless for this investigation: your profiler and disassembler cannot attribute samples to function names without symbols, and you cannot map an assembly instruction back to a source line. Build (or keep) an unstripped copy with the exact same optimization flags as the suspect build, and profile that copy.

Compare the generated assembly for the hot function

For Go, go build -gcflags=-m prints the compiler's own inlining decisions:

./main.go:3:6: can inline abs
./main.go:13:15: inlining call to abs

If that "inlining call to abs" line disappears after the flag change, that is your regression, confirmed directly from the compiler's own output. When the -m output does not fully explain it, go one level deeper: go tool compile -S (or objdump -d for other toolchains, Compiler Explorer for a quick C/C++/Rust comparison) dumps the generated assembly. Look for a shrunk instruction count that grew back (something that used to be inlined or vectorized, meaning the compiler combined several operations into one wide CPU instruction, no longer is) or new bounds checks (a runtime check the compiler inserts before an array access to make sure the index is in range) and register spills (the compiler ran out of fast CPU registers and had to store a value in slower memory instead) that were previously eliminated.

Mitigate in CI/CD going forward

Pin the compiler version and flag set explicitly rather than inheriting "latest" from a base image; treat a flag change like a dependency bump, reviewed the same way. Add a benchmark regression gate to CI: run the same hot-path microbenchmarks this investigation used, compare against a stored baseline, and fail the build if a defined tolerance (for example 5%) is exceeded, so this class of regression is caught before merge instead of after a fleet-wide rollout. Canary any flag change to a small percentage of the fleet with the same before/after CPU and latency comparison used above before rolling it out everywhere.

What Amdahl's Law says

Amdahl's Law caps the speedup you can get from optimizing part of a system by how much of the TOTAL time that part actually represents, no matter how much faster you make it. Speeding up a small piece of the system, however dramatically, can only ever move the total by a small amount.

The formula

Speedup = 1 / ((1 - p) + p / s)

Here p is the fraction of total execution time the optimized component represents, and s is how many times faster that component becomes. As s approaches infinity ("infinitely fast"), the p / s term goes to zero, so the formula simplifies to a ceiling:

Speedup_max = 1 / (1 - p)

Worked example

The component accounts for p = 0.30 (30 percent) of execution time. Making it infinitely fast:

Speedup_max = 1 / (1 - 0.30) = 1 / 0.70 ~= 1.43x

So even a perfect, instantaneous fix to that 30 percent only speeds the WHOLE system up by about 43 percent, because the untouched 70 percent now dominates total runtime.

For comparison, a realistic (not infinite) 10x speedup on that same component gives:

Speedup = 1 / ((1 - 0.30) + 0.30 / 10) = 1 / (0.70 + 0.03) = 1 / 0.73 ~= 1.37x

That's already close to the 1.43x ceiling, which is the important part: once a component is sped up enough, chasing further speedup on it has rapidly diminishing returns.

How this informs prioritization

It tells you the maximum POSSIBLE payoff of optimizing a given piece before you spend engineering time on it, so you can rule out low-p targets immediately (a function that's 5 percent of runtime caps your total win at roughly 5 percent, no matter how cleverly it's rewritten) and focus effort on whatever currently has the highest p. It's also a guard against sunk-cost optimization: once a component's remaining time share is small, further work there yields little, better to re-profile and find the NEW dominant cost than keep polishing something that's already stopped mattering much.

What profiling is

Profiling is measuring where a running program actually spends its CPU time, memory, I/O (input/output, e.g. disk and network operations), or network calls, instead of guessing. In Site Reliability Engineering (SRE, the discipline of keeping production systems reliable through engineering practice), profiling turns "the API feels slow" into a data-backed answer: which function, query, or syscall is actually consuming the resource.

Goals of profiling

• Locate the dominant cost, not every cost. The aim is to find the one or two paths that account for most of the time or memory, because fixing those gives the biggest return.
• Separate symptom from root cause (high CPU might really be garbage collection thrashing, not your business logic).
• Produce evidence you can attach to a change request, instead of an opinion.

Sampling vs instrumentation profilers

• Sampling profilers interrupt the program at a fixed interval (say every 10ms) and record which function is on-CPU at that instant. Over thousands of samples, functions that show up often are the hot ones. Low overhead, safe to run in production, but statistical, very short functions can be under-sampled.
• Instrumentation profilers insert timing code around specific calls, giving exact counts and durations. Precise, but adds real overhead since every instrumented call now does extra work, so it is usually run in staging or narrowed to one deliberately chosen section in production.

Metrics typically collected

• CPU: time-on-CPU per function or thread, often visualized as a flame graph.
• Memory: allocation rate, heap size over time, retained objects.
• I/O: disk read/write latency and throughput, syscall counts.
• Network: request/response latency, bytes transferred, connection counts.

Worked example

A checkout API's p95 latency (95th percentile response time, meaning 95% of requests finish faster than this number) rises from 120ms to 300ms. A CPU sampling profile taken during the slow window shows 70% of samples land inside one function, calculate_discount(). That single number tells you exactly where to look before touching any code.

Why measure-first, change-later

Without a profile, "optimizing" is guessing: you might rewrite a function that is only 2% of runtime and see no improvement, while the real 70% offender goes untouched. Measuring first also gives you a baseline to prove a fix actually worked, and it keeps each change small and reversible if the hypothesis turns out wrong, which fits SRE's broader preference for low-risk, evidence-driven production changes over large speculative rewrites.

Before-and-after measurement plan

Re-run the SAME benchmark/harness used to establish the original baseline, under matched conditions: same load shape, same environment, and the same time-of-day window if traffic has a daily pattern. The change under test should be the only thing that's different between the two runs. Take multiple repetitions on each side, not one run each.

Statistical significance

Single numbers are noisy, so compare distributions instead of points. A practical, interview-safe approach: run N repetitions of before and after, compute the mean and spread (e.g. standard deviation) for each, and check whether the difference is bigger than the natural run-to-run variance, either with a simple significance test (a t-test, a standard statistical check for whether two averages differ by more than you'd expect from random chance) or, since latency data is usually skewed (bunched to one side with a long tail of slow outliers) rather than normally distributed (evenly spread in the familiar bell-curve shape), by comparing medians/percentiles directly or bootstrapping a confidence interval on the difference (repeatedly resampling the data to estimate a likely range for the true difference).

Concretely: if before averages 200ms with a spread of about 5ms (a typical range of roughly 195-205ms across repetitions) and after averages 180ms with a spread of about 5ms (roughly 175-185ms), the two ranges sit fully apart, a clear 10ms gap between the top of the after range and the bottom of the before range, so that's a real improvement. If the two ranges overlap heavily instead, for example if the after spread widened to +/-15ms (165-195ms, swallowing most of that gap), you cannot yet claim a win, you need more runs or a bigger effect.

Regression analysis for non-target metrics

A fix aimed at one metric (say, CPU time) can silently regress another (memory, error rate, or a different endpoint sharing the changed code path). Re-run the FULL metric dashboard before and after, not just the metric you optimized, paying particular attention to anything else that shares the changed component.

Communicating to stakeholders

Report: what metric moved and by how much, WITH its uncertainty or range rather than a single point number; the conditions it was measured under (e.g. "under X requests/sec of synthetic load" versus "confirmed against live production traffic"), so the audience knows the claim's scope; what else was checked and found unaffected; and what will be watched going forward (e.g. an alert threshold tied to the improved metric). Avoid presenting one lucky run as proof, since that's exactly the trap this whole validation process exists to avoid.