Performance Cost Optimization & Resource Efficiency Questions

Optimizing for the money and resources a given level of performance consumes, not just raw speed. Covers cost-per-request reasoning, right-sizing compute and memory, efficiency of resource utilization, and trading performance against spend. Emphasizes treating cost and resource efficiency as first-class performance objectives.

HardTechnical

87 practiced

Propose an instrumentation and sampling strategy to attribute end-to-end tail latency across 5 microservices at 100k RPS. Discuss how to separate compute, network, blocking I/O, and GC contributions and how to store sampled traces without incurring prohibitive cost.

Sample Answer

Requirements & constraints:- 100k RPS across 5 microservices, need end-to-end attribution for tail latency (99.9/99.99).- Must separate compute, network, blocking I/O, and GC contributions.- Keep tracing cost low while preserving fidelity for tail events.

High-level approach:- Use distributed tracing (OpenTelemetry-compatible) with consistent trace-id propagation across services, and high-resolution epoch timestamps at span ingress/egress.- Combine lightweight continuous sampling with triggered high-fidelity capture when tail conditions are detected.

Instrumentation details:- Spans: instrument each RPC boundary (ingress/egress) and key internal operations (thread-pool queueing, DB call, filesystem call). Record: event timestamps, thread id, cpu-time delta (see below), bytes in/out, socket RTT when available.- CPU vs walltime: capture per-span CPU-time using runtime APIs (clock_gettime(CLOCK_THREAD_CPUTIME_ID) or thread CPU counters) at span start/end. CPU delta isolates compute from blocking.- Blocking I/O: instrument blocking libraries (DB client, HTTP client, file I/O) to annotate spans as “blocking” and record wait-start/wait-end timestamps. For languages without hooks, use async wrappers or runtime thread-block detection (e.g., tracked idle time).- Network: record socket-level metrics (connect time, TLS handshake, write/read syscall durations). Optionally use eBPF on hosts to measure kernel-level network latency and tag by conn tuple -> correlate with trace-id hashed into socket metadata.- GC/Stop-the-World: subscribe to runtime GC events (JVM: GC notification, Go: runtime/metrics). When a GC pause overlaps a span, annotate span with gc_pause_ms and subtract from walltime to attribute to GC.- Queueing/Contention: record queue-enter/leave for thread pools; measure queue time separately.

Sampling strategy:- Two-tier sampling: 1. Base sampling: low-rate uniform sampling (e.g., 0.1% of traces) for broad observability. 2. Adaptive tail-triggered sampling: lightweight local detectors at ingress/egress measure observed latency percentiles per-second (sliding window). If an incoming request exceeds a threshold (e.g., 95th or 99th compared to local baseline) or if downstream service reports high latency, mark trace for full capture. Also trigger full capture for traces with anomalies: high CPU delta, GC overlap, socket retransmits.- Probabilistic reservoir for partial traces: if tail detector fires downstream, request upstream services to retroactively increase sampling using “trace-sampling-adjust” header for a short TTL (propagated via trace-id).- Profiling sampling: continuously run very-low-rate (e.g., 0.01-0.1% of requests) CPU/stack sampling (async profiler or eBPF) and correlate profile samples to trace-ids for root-cause without full-trace capture every time.

Storage & cost control:- Store full high-fidelity traces only for sampled/tail-triggered traces. Compress spans (protobuf + gzip), store in object store (S3) with lifecycle rules.- Index minimal metadata for every trace (trace-id, start-time, latency, service-level summaries, sampling-reason) in a small indexed DB (Dynamo/ElastiCache) to look up traces for tail percentiles cheaply.- Aggregate metrics (histograms, percentiles) continuously at edge and store rollups; use these to detect tail and trigger captures.- Retention: keep full traces for short window (7-30 days) and aggregated summaries for longer.

Analysis & attribution:- For each sampled trace compute: - wall_time_total = sum(span wall durations) - cpu_time_total = sum(span cpu deltas) - io_block_total = sum(spans marked blocking) - gc_total = sum(gc_pause_ms overlapping trace) - network_total = sum(socket-level network durations + eBPF-observed kernel latencies) - queue_total = sum(queue wait times- Attribution: present as stacked waterfall per-trace and aggregated percentiles; use causality (span tree) to attribute time to specific services/components.

Trade-offs & risks:- Overhead: keep in-process instrumentation minimal; push heavy measurements (eBPF, profiles) to dedicated sidecars or sampling agents.- Detection latency: local detectors can miss cross-service cascades; use short TTL retro-sampling header to catch downstream tails.- Complexity: eBPF + runtime hooks increase operational surface; prefer runtime APIs where available and fall back to network-level telemetry.

This strategy balances cost and fidelity: low continuous sampling for broad health + high-fidelity capture for tails and targeted profiling to root-cause compute, network, blocking I/O, and GC contributions.

HardTechnical

95 practiced

Design storage layout for a time-series metrics system ingesting 5k series at 1k samples/sec each, retention 2 years, and requirement that queries for last 24 hours return under 1 second. Discuss compression, downsampling, partitioning, and expected storage cost trade-offs.

Sample Answer

Requirements recap: ingest 5,000 series × 1,000 samples/sec = 5M samples/sec; retention = 2 years; interactive queries for “last 24h” must return < 1s. Key constraints: extremely high ingest, large retention window, low-latency recent queries.

High-level design- Use a distributed time-series storage (append-optimized TSDB) with hot (recent) and cold (long-term) tiers.- Shard by time + series hash: create time-based buckets (e.g., 1h shards) and hash series id across nodes within each time shard to balance writes.- Keep recent 24–72 hours in hot tier (SSD/NVMe + in-memory index). Older data moved to cold tier (object storage / HDD) with compacted files (columnar / block format).

Compression- Use time-series encodings (Gorilla-style XOR for values, delta/delta for timestamps) achieving 5–10× on floats and timestamps.- Store blocks (e.g., 64KB - 1MB) with per-block min/max and bloom/bitmap indexes to enable skip scans.- Compress blocks with fast codecs (LZ4/Zstd-1) for fast decompression during queries.

Downsampling & retention policy- Keep raw data for short period (e.g., 7–30 days) in hot/cold raw form for high-fidelity analysis.- Create downsampled rollups (1m, 5m, 1h aggregates) stored for entire 2-year retention. Aggregates include count, sum, min, max, maybe stddev.- Use deterministic hierarchical rollups produced in streaming (e.g., in ingest pipeline) to avoid reprocessing.

Partitioning & indexing for low-latency 24h queries- Partition by time shard (hour) to limit files scanned to last 24 shards (or fewer if larger shards).- Within shards, use series-level partitioning (series hash) so a single series query goes to one node/shard.- Maintain an in-memory/SSD index for the hot 24h window: per-series pointers to blocks in time order plus bloom filters to avoid IO.- Support query fan-in: parallel scan of the 24h shards across nodes; use block-level pruning via time-range and value indexes to keep latency <1s.

Storage & cost calculation (example)- Assume raw sample ~16 bytes (8B timestamp, 8B float). Ingest = 5M *16B = 80MB/s ≈ 6.9TB/day raw.- With 8× compression (Gorilla + blocking) → ~0.86TB/day.- Raw retention 30 days → 25.8TB. Remaining 23 months stored as 1m aggregates: 5k series × (60*24*365*2 / 1) ≈ 5k * 1,051,200 ≈ 5.26B points; if aggregate stored as 24B → ~126GB per year? (note: recalc carefully for client). Practical estimate: downsampling reduces multi-year cost by 10–60×.- Cold tier on S3/Blob + lifecycle policies yields lowest $/GB; hot SSD cluster sized to hold 24–72h compressed raw + indexes (e.g., few TBs).

Trade-offs- More raw retention => higher fidelity but much higher cost and larger hot storage; affects query latency if not well indexed.- Higher compression ratio lowers storage cost but increases CPU and decompression latency; choose fast codecs and smaller block sizes for low-latency reads.- Aggressive downsampling reduces storage dramatically and simplifies long-term queries, but loses granularity and prevents some analyses.- Finer time-shards improve parallelism and compaction but increase metadata and management overhead.

Operational notes- Autoscale ingestion nodes and shard placement; use backpressure and batching.- Continuous background compaction to merge small blocks and build rollups.- Monitor query P95 and IO; keep the last-24h index memory budget to meet <1s SLAs.

This design balances cost and latency by keeping recent data hot and indexed for sub-second queries, using time-series-specific compression, and moving to downsampled cold storage for long retention.

MediumTechnical

105 practiced

A client is billed high network egress costs for serving user downloads across regions. As a Solutions Architect, propose strategies to reduce egress cost and explain trade-offs between user latency, replication cost, and data consistency.

Sample Answer

Situation: A client faces high network egress charges delivering downloadable assets to users across multiple regions. They want lower cost without unacceptable impact to user experience.

Proposal (strategies):1. Use a CDN as primary mitigator - Put assets on CDN with long TTLs, Cache-Control immutable for versioned files. - Enable origin shielding to reduce origin fetches. Trade-off: CDN edge hits reduce egress and latency; cache misses still incur origin egress.

2. Regional replication of storage (multi-region buckets / geo-replication) - Replicate objects to one or two hub regions close to major user populations. - Use routing (GeoDNS / Traffic Manager) to direct users to nearest replica. Trade-off: Replication increases storage and inter-region transfer costs, but reduces cross-region egress per request and improves latency.

3. Bring-your-own-edge / peer-to-peer/offload - Use edge caches or third-party peering that has cheaper egress pricing. Trade-off: Operational complexity and vendor lock-in.

4. Smart caching + compression + range support - Pre-compress assets, enable Brotli/GZIP, support range downloads to avoid re-sending full files. Trade-off: Slight CPU/storage overhead vs lower bandwidth.

5. Signed URLs + cache busting for versioned downloads - Allow long caching for immutable files; only new versions cause invalidation. Trade-off: Complexity in URL generation; reduces frequent cache churn.

6. Analyze and optimize traffic patterns - Use analytics to find hot objects and pre-warm CDN or replicate only hot items. Trade-off: Requires monitoring and automation but minimizes replication cost.

Consistency and latency trade-offs:- Strong consistency (single writable region) simplifies writes but forces reads from that region or requires synchronous replication — increasing replication cost and latency for distant users.- Eventual consistency (async replication) lowers write latency/cost and lets reads be served locally, reducing egress and improving user latency, at the cost of temporary read-after-write anomalies.Recommendation: For static, immutable downloadable assets use CDN + versioned objects with eventual replication to regional storage for hot regions. For frequently updated assets where read-after-write correctness matters, prioritize stronger consistency in a single region and accept higher replication or egress cost, or implement application-level read-after-write routing.

Quantify & next steps:- Measure percent of origin egress vs edge hits, top 10 objects by bandwidth.- Estimate CDN vs inter-region egress pricing for their cloud provider and run cost scenarios (no CDN, CDN-only, CDN+regional replication).- Prototype CDN + one regional replica for largest region and compare cost/latency.

MediumSystem Design

100 practiced

Design an SLO-based alerting strategy to detect performance regressions before users are impacted for a service with 99.95% availability and p95 latency 200ms. Specify metrics, alert windows, escalation policies, and how to avoid noisy alerts.

Sample Answer

Requirements & constraints:- SLOs: availability 99.95% (≈ 22 min/month allowable downtime) and p95 latency ≤ 200ms.- Detect regressions early (before user impact), low false positives, scalable across services.

High-level approach:- Define Service-Level Indicators (SLIs) derived from production telemetry, compute SLO burn rates, and create multi-tier alerts: Burn-rate, SLI-threshold, and symptom/perf-degradation alerts.

Key metrics & SLI definitions:- Availability SLI: ratio of successful requests (HTTP 2xx/3xx) over total eligible requests (exclude known maintenance) in rolling windows.- Latency SLI: p95 request latency for user-facing API endpoints (client-observed when possible).- Error budget: 100% - SLO (0.05%) tracked over 30d rolling window.

Alert windows & thresholds:1. Early-warning (burn-rate) — high priority for regression detection: - Compute burn rate over short windows: 5m, 30m, and 6h. - Trigger when burn_rate > 3x for 5m OR > 2x for 30m. (Example: if 5m error rate consumes >3x expected error budget). - Purpose: surface sudden regressions before SLO breach.

2. SLO-proximity: - Warning: projected monthly availability loss > 20% of remaining error budget computed over 6h window. - Critical: projected burn exceeds remaining budget (i.e., SLO projected to miss) over 24h projection.

3. Symptom alerts (direct user impact): - p95 latency > 200ms sustained for 5m (warning) and 15m (critical). - Error rate absolute > 0.5% for 5m (tunable per service traffic).

Escalation policy:- Warning alerts → paging to on-call via low-urgency channel (Slack + mobile push), require acknowledgement within 15m.- Critical/burn-rate triggers → immediate pager to primary on-call, escalate to secondary after 5–10m if unacked, and notify SRE lead and product owner on persistent 30m incidents.- Include runbooks with triage steps, dashboards, and rollback/playbook links.

Avoiding noisy alerts:- Use composite alerts: require both SLI degradation and upstream dependency health change OR increased latency + increased error budget burn to fire critical page.- Use multi-window confirmation (e.g., 2 out of 3 windows exceed threshold) to avoid flapping.- Rate-limit/aggregate alerts per deployment id and per cluster; suppress during planned maintenance via automatic maintenance windows from CI/CD.- Apply statistically-significant checks: require >X requests in window before considering p95 valid.- Use auto-snooze for repeated identical alerts if acknowledged and mitigation in progress, with periodic reminders.

Instrumentation & tooling:- Export request-level traces, metrics to Prometheus/Tempo/Grafana or vendor observability (Datadog/NewRelic).- Compute SLIs with sliding-window aggregation (Prometheus recording rules or metrics backend).- Dashboards: SLO burn, traffic, p95 heatmaps, top endpoints by latency/errors, recent deploys.- Integrate alerts with CI/CD to correlate deploys with regressions; auto-open incident in tracker with links.

Trade-offs & rationale:- Burn-rate alerts detect fast regressions without waiting for SLO breach.- Multi-window and composite logic reduce false positives.- Projection windows balance sensitivity vs noise; tune thresholds per service traffic patterns and customer SLAs.

Operationalization:- Start conservative thresholds in production, run for 2–4 weeks, measure precision/recall of alerts, and iterate thresholds and runbook actions with ops feedback.

EasyTechnical

88 practiced

A customer expects 2k RPS average and 8k RPS peak. Each request consumes about 15ms of CPU and requires 1 MB of working memory while processing. A VM offers 4 vCPU and 8 GB memory and costs $0.08/hr. As a Solutions Architect, estimate number of VMs for average and peak with 30% headroom and calculate monthly cost. Show assumptions.

Unlock Full Question Bank

Get access to hundreds of Performance Cost Optimization & Resource Efficiency interview questions and detailed answers.

Join thousands of developers preparing for their dream job.