Performance Trade-offs & Optimization Strategy Questions
Deciding what to optimize, how far, and at what cost to other qualities. Covers performance vs readability/reliability/cost trade-offs, prioritizing the optimization with the highest payoff, knowing when a system is fast enough, and sequencing optimization work. Emphasizes optimization as a strategic engineering judgment rather than a reflex.
EasyTechnical
68 practiced
You need a data structure to maintain per-user metric aggregates over a sliding window (for example, last 5 minutes) with millions of users. Describe suitable data structure choices, memory and CPU trade-offs, and sketch an approach that supports insert(timestamp, user, value) and query(user, window) efficiently for SRE monitoring at scale.
Sample Answer
Situation: We need per-user metric aggregates over a sliding window (e.g., last 5 minutes) for millions of users with high write throughput and low-latency queries.Recommended approach (practical, memory-efficient, exact):- Per-user fixed-size circular bucket buffer. Choose bucket width b (e.g., 10s) and W = window/b (e.g., 30 buckets for 5 minutes).- Data structure: hashmap from userID -> struct { lastSeenEpoch, buckets[W] }, where each bucket stores (bucketTimestamp, sum, count, maybe min/max).- insert(timestamp, user, value): - Lookup or create user's struct (create lazily). - Map timestamp -> bucket index = floor(timestamp/b) % W. - If bucketTimestamp != current bucket's epoch, zero it and set bucketTimestamp. - Add value to sum and increment count. - O(1) per insert.- query(user, window): - Sum buckets whose bucketTimestamp >= now - window. O(W) per user (typically small, e.g., 30). - Compute average, rate, etc.Memory & CPU trade-offs:- Memory per user ≈ W * size(bucket) + hashmap overhead. Example: W=30, bucket = {ts:8,sum:8,count:8} ~ 24 bytes → ~720B/user + map overhead → for 10M users → ~7.2GB + hashmap cost. Tune W, b, and stored fields to control memory.- CPU: inserts O(1); queries O(W). If queries are frequent, pre-aggregate sliding sum by maintaining per-user runningWindowSum that is updated when buckets rotate (amortized O(1)), turning queries into O(1).Scaling & optimizations:- Shard by userID across multiple machines/processes to distribute memory/CPU.- Evict idle users: track lastSeenEpoch and TTL (e.g., drop users idle > window * k).- Approximate options if memory constrained: use Count-Min Sketch per-shard for approximate sums (low memory, no per-user exact), or use probabilistic reservoir/T-digest for distribution summaries.- Use batching and efficient serialization for persistence; keep in-memory for hot users, cold data in blob store.- Concurrency: per-shard locks or lock-free atomic updates to buckets; prefer single-writer per-shard for simplicity.Why this works for SRE:- Predictable memory and bounded per-user work.- Fast writes (O(1)) and low-latency queries (O(W) or O(1) with rolling sum).- Easy to shard, evict, and switch to approximate methods when scale/memory requires trade-offs.
MediumTechnical
108 practiced
Explain the trade-offs between synchronous and asynchronous replication for a database used by an e-commerce platform. Discuss write latency, durability guarantees, failover complexity, consistency, throughput, and cost. From an SRE perspective, which factors weigh most for cart checkout transactions versus analytics workloads?
Sample Answer
Synchronous replication: the primary waits for replicas to acknowledge a write before returning success. Asynchronous replication: primary returns once local commit is done; replication to replicas happens later.Trade-offs by dimension:- Write latency: synchronous increases client-visible latency (depends on replica RTT); async has lower write latency.- Durability guarantees: sync gives stronger durability (no acknowledged commit lost if primary dies and replicas confirmed); async risks acknowledged writes not yet shipped—possible data loss.- Failover complexity: sync simplifies safe failover (promoted replica is up-to-date); async requires crash-consistent recovery, log replay, and potential data loss reconciliation.- Consistency: sync provides stronger linearizability across replicas; async yields eventual consistency and read anomalies.- Throughput: async typically allows higher throughput since primary isn’t blocked; sync can throttle throughput when waiting for acknowledgements.- Cost: sync may require more expensive networking (low-latency links), more replicas in close regions; async can leverage cheaper, geographically dispersed replicas.SRE perspective — priorities:- Cart checkout (critical OLTP): favor durability and consistency over minimal latency. Use synchronous (or hybrid: local sync, remote async), multi-zone sync with fast links, strict monitoring of replication lag, tight SLOs for commit latency, automated failover with health checks and leader election to avoid split-brain.- Analytics (OLAP): favor throughput, cost, and geographic distribution. Use asynchronous replication to offload reads, accept eventual consistency, larger replication lag tolerated, focus on batch correctness, cheaper long-term storage, and simple read replica scaling. Monitor lag trends, alert on sustained growth, and have re-sync procedures.Recommendation: adopt hybrid model—sync for critical hot-path writes (checkout, payments) and async for analytics/backups. Define per-workload SLOs, test failovers, and instrument replication metrics to drive operational decisions.
HardTechnical
59 practiced
Given cloud instance pricing and a CPU-bound workload, model the trade-off between buying larger instances (higher per-hour cost but fewer instances) and optimizing code to reduce CPU consumption by 30%. Outline how you would quantify cost savings, benchmark methodology, pricing considerations (e.g., reserved vs on-demand), and risks of regressions from code changes.
Sample Answer
Goal: compare cost of buying larger instances vs investing engineering time to optimize CPU by 30% for a CPU-bound workload. I’ll outline a quantitative model, benchmarking plan, pricing considerations, and regression risk controls.1) Quantitative model- Measure baseline: N_small instances, each with CPU_c capacity, cost P_small/hour, throughput T_total (requests/sec) or work units/hour.- If a larger instance has k× CPU (k≈ cores×clock) and cost P_large, needed instances = ceil(N_small / k). Hourly cost_small = N_small * P_small; cost_large = N_large * P_large.- Optimization: 30% CPU reduction means effective capacity per instance increases by 1/(1-0.3)=~1.43x, so required instances = ceil(N_small / 1.43). Compute cost_opt = N_opt * P_small (or P_large if migrating).- Compare: savings_opt = cost_baseline - cost_opt; savings_size = cost_baseline - cost_large. Compute ROI: engineering_hours * hourly_rate vs annualized infra savings.Example: baseline 10 x $0.10 = $1/hr. Large is 2× CPU at $0.22 -> need 5 -> $1.10/hr (worse). With 30% opt need 7 -> $0.70/hr -> $0.30/hr saved.2) Benchmark methodology- Create representative, stable workload (synthetic or replayed production traces).- Use performance harness that measures CPU utilization, latency percentiles, throughput, and error rates.- Run each scenario: baseline code on small instances, optimized code on small, baseline on large instances. Repeat runs (≥5) to get CI, warm caches, remove autoscaling events.- Capture system metrics (proc CPU, steal, syscall rates), tail latency (p99/p999), and power/cost meters.- Use statistical tests (t-test or bootstrap) for significance.3) Pricing considerations- On-demand vs reserved: reserved or savings plans lower effective P by committed term—include amortized reservation cost when modeling long-term savings. Spot instances reduce cost but add preemption risk—model additional SRE effort and risk buffer.- Instance sizing granularity: CPU vs vCPU vs dedicated cores—measure actual single-thread vs multi-thread scaling.- Network, memory, and licensing costs: optimization could shift resource profile; ensure not increasing other costs (e.g., more memory usage).4) Risks of regressions from code change- Introduce comprehensive test matrix: unit, integration, perf regression tests, and canary rollout (5–10%) with monitoring for error rates, latency, resource usage.- Maintain benchmark baselines and automated CI perf gates (fail on >X% regressions).- Plan rollback playbook and instrumentation to trace regressions (distributed tracing, flamegraphs).- Consider non-performance side-effects: correctness, maintainability, security.5) Decision framework- Compute payback period = engineering_hours * cost_rate / hourly_savings. If payback < threshold (e.g., 6 months) and regression risk manageable, optimize code; otherwise choose instance resize or mixed approach (right-size + modest optimization).- Document assumptions and rerun quarterly as pricing or workload changes.This approach quantifies savings, uses reproducible benchmarks, factors pricing commitments, and minimizes regression risk through staged rollout and automated perf gates.
MediumTechnical
58 practiced
Describe how you would use Linux perf, flamegraphs, and eBPF tools to diagnose kernel-level and user-space bottlenecks in a high-throughput web server. Explain setup for production-safe collection, sampling intervals, overhead trade-offs, and how to correlate kernel and user-space findings.
Sample Answer
Situation: I’d approach this as an SRE diagnosing a sustained high-throughput web server showing latency/p95 regressions. Goal: identify whether CPU, kernel (syscalls, locks, I/O), or user-space code is the bottleneck, using production-safe sampling and correlate results across kernel/user space.Plan and tools:- perf: low-level sampling profiler that captures kernel+user stacks (perf record -F <hz> -p <pid> -g -- sleep <dur>)- Flamegraphs: use Brendan Gregg’s perf script → stackcollapse-perf.pl → flamegraph.pl to visualize hot paths- eBPF (bcc / bpftrace / libbpf-tools): for low-overhead continuous metrics, syscall traces, off-CPU stacks, and histograms (e.g., offcputime, biolatency, runqlat)Production-safe collection:- Run sampling (not tracing every event). Typical sampling rates: 49–199 Hz. Start ~99 Hz; reduce to 49 Hz if overhead must be minimal. Higher Hz (>=400) increases fidelity but also overhead.- Short continuous captures: e.g., 30–60s perf recordings during issue windows. For longer baseline, run eBPF histograms (aggregated) rather than perf record.- Use perf_event_paranoid adjustments or run as root via controlled automation. Prefer ephemeral collectors (ssh+tunnel) and store profiles off-box.- In containers: mount /sys and enable CAP_SYS_ADMIN or use node-level collectors. Use flamegraphs offline to avoid production CPU spikes.Commands/examples:- Kernel+user sampling (30s at 99Hz): perf record -F 99 -p <PID> -g -- sleep 30 perf script | ./stackcollapse-perf.pl > out.folded ./flamegraph.pl out.folded > perf.svg- Capture system-wide stacks: perf record -a -F 99 -g -- sleep 30- eBPF quick histogram (bcc): sudo /usr/share/bcc/tools/offcputime -p <PID> 30 > offcpu.txt- Trace syscalls with bpftrace (low overhead aggregated): sudo bpftrace -e 'tracepoint:syscalls:sys_enter_* { @[comm, probe] = count(); }' -d 60Overhead trade-offs:- perf sampling: moderate overhead mostly proportional to sample rate; inlined stacks vs. unwinders (unwinder adds cost). Use frame-pointer-based unwinding when available; enable -g.- eBPF aggregations: very low overhead for histograms and counters; avoid per-event logging.- Instrumentation (log/tracing): highest overhead — avoid full traces in prod.Correlating kernel and user-space findings:- Ensure perf records both kernel and user stacks (-k/-u flags default). When you see hot kernel functions (e.g., submit_bio, tcp_sendmsg, futex_wait), map them to the user stacks in the same perf flamegraph — flamegraphs will show call stacks crossing user→kernel (if folded to include kernel frames).- Use per-thread PID/TID tagging: perf script emits pid/tid; include them in folded output to join with application logs/trace IDs.- Use eBPF offcputime to find where threads spend time sleeping in kernel and pair with perf user-space CPU hotspots to tell if latency is due to scheduling/waits or user CPU work.- For IO or network, run bpftrace/bcc tools that capture syscall latencies (biolatency, tcpconnect, tcplife) and correlate timestamps with perf samples or logs.- If results are ambiguous, capture perf.map for JIT languages / symbolization (e.g., Java) and use addr2line / pprof stacks.Result pattern examples:- If flamegraph shows user CPU hot path in request handler → optimize algorithm or add CPU capacity.- If kernel hot path shows tcp_sendmsg or lock contention → tune network stack, socket buffers, or reduce syscalls (batching) and use epoll/reactor changes.- If off-CPU shows futex or schedule → investigate lock contention, GC pauses, or thread starvation.Final note: start with low-frequency perf or eBPF aggregated captures, iterate with slightly higher fidelity when you reproduce the issue, always test collection overhead in staging, and automate safe collectors for postmortems.
MediumTechnical
57 practiced
Design an experiment to validate that a proposed optimization reduces CPU usage by at least 10% without increasing P99 latency. Define the control and treatment setup, warmup period, metrics to collect, required sample size or power calculation, statistical test, and rollback criteria for production rollout.
Sample Answer
Situation & goal: Validate that an optimization reduces CPU usage by ≥10% without increasing P99 latency (no regression). We'll run a randomized online experiment (canary/A–B) with clear instrumentation, stats plan, and rollback criteria.Control & treatment:- Control: current production binary/config on group A.- Treatment: optimized version on group B.- Traffic split: 50/50 (or weighted if riskier). Randomly assign requests/hosts.Warmup & duration:- Warmup: 10 minutes after rollout per host to let JIT/caches/GC stabilize (or N warmup requests/containers).- Run for at least the time needed to collect planned sample size (see below); ensure coverage of typical traffic patterns (include peak window, at least 1 full business day).Metrics to collect (sampled at 1-min granularity and per-request traces):- Primary: - Mean CPU% per host (1-min aggregates) - P99 request latency (end-to-end) per minute and raw traces- Safety: error rate (5xx), throughput (rps), GC/pause metrics, queue lengths, instance churn- Context: request mix, payload size, instance typeSampling unit & sample size (power calc example):- Unit: 1-minute host-level CPU% aggregates. Baseline mean μ0=60%, σ≈8% (estimate from monitoring). Want Δ = 10% of μ0 = 6% absolute reduction.- Two-sided t-test, α=0.05, power=80% (β=0.2): n ≈ 2*(Zα/2+Zβ)^2*σ^2/Δ^2 = 2*(1.96+0.84)^2*8^2/6^2 ≈ 28 minutes per group (hosts × minutes). If using fewer hosts, extend minutes; if CPU variance higher, increase n. Capture more for safety (recommend ≥60 samples per group).- P99: treat as non-inferiority check. P99 estimates have heavy tails; collect many requests (recommend ≥100k requests per group) and use bootstrap to estimate CI.Statistical tests:- CPU: two-sample t-test (or Welch’s t-test) on minute-aggregates; report mean difference and 95% CI. Success if mean CPU reduction ≥6% and p<0.05.- P99 latency: non-inferiority test using bootstrap CIs on difference (treatment - control). Define non-inferiority margin (e.g., +5ms or +2% relative). Success if upper bound of 95% CI ≤ margin.- Also check error rates (chi-square test) and throughput stability.Multiple comparisons & robustness:- Pre-register primary/secondary metrics. Adjust for multiple testing (Bonferroni or hierarchical testing) if needed.- Run sensitivity: stratify by region, instance type, request path.Rollback / launch criteria:- Immediate rollback during experiment if: - Error rate increases >0.1 percentage points or by >50% relative - P99 increases beyond non-inferiority margin in two consecutive 5-minute windows - Throughput drops or instance crashes occur- Post-experiment promotion if: - CPU mean reduction ≥10% with p<0.05 - P99 non-inferiority satisfied - No safety metric regressions and SLOs not violated for 24-hour extended canaryObservability & runbook:- Dashboards for CPU, P99, errors, GC, and hosts; alerting on rollback thresholds; automated rollback pipeline to flip traffic and deploy previous version.This plan balances statistical rigor with operational safety; adjust σ, Δ, and sample unit based on real telemetry and repeat or increase sample size if P99 variance is high.
Unlock Full Question Bank
Get access to hundreds of Performance Trade-offs & Optimization Strategy interview questions and detailed answers.
Sign in to ContinueJoin thousands of developers preparing for their dream job.