Performance Cost Optimization & Resource Efficiency Questions

Optimizing for the money and resources a given level of performance consumes, not just raw speed. Covers cost-per-request reasoning, right-sizing compute and memory, efficiency of resource utilization, and trading performance against spend. Emphasizes treating cost and resource efficiency as first-class performance objectives.

HardSystem Design

101 practiced

Design a sharding strategy for a user-profile store that must scale to billions of records while supporting low-latency lookups and occasional global analytics. Discuss shard-key selection, range vs hash sharding, resharding approaches (online vs offline), consistent hashing, metadata service for routing, rebalancing algorithms, and minimal-downtime migration patterns.

Sample Answer

**Clarify requirements**- Low-latency point lookups (user profile by id), scale to billions, occasional global analytics (full-scan/aggregates), strong read availability, acceptable eventual consistency for rebalances.

**Shard-key selection**- Primary: user_id (stable, evenly distributed). Avoid write-hot keys (e.g., region) unless needed.- Add secondary indexes or materialized views for analytics.

**Range vs Hash sharding**- Hash sharding on user_id for uniform distribution and predictable lookup latency.- Range sharding useful if you need range queries (time-ordered), but risk hotspots and complex rebalancing.

**Consistent hashing & metadata service**- Use consistent hashing (virtual nodes) to map user_id hash -> shard to minimize key movement on reshard.- Metadata service (small, highly-available service or config in ZooKeeper/Consul) stores mapping of vnode -> physical shard and exposes routing API and cacheable config to app servers.

**Resharding approaches**- Online: preferred. Add empty shards, update metadata, use consistent-hash ring; run background bulk movers copying keys from source shards to new shards while serving reads/writes (dual-write or read-redirect).- Offline: take shards down—only for maintenance windows; high downtime.

**Rebalancing algorithms**- Greedy load-based: move vnodes from overloaded nodes to underloaded nodes using metrics (CPU, storage, QPS). Prioritize moving minimal vnodes to reduce churn.- Throttling with rate limits, backpressure, and health checks.

**Minimal-downtime migration patterns**- Dual-write: app writes to old and new shard (via middleware) while background copy completes; reads directed to new once catch-up verified.- Read-repair / tombstone: route reads to new; if missing, read from old and async copy.- Use versioned metadata and rolling update of routing caches; ensure atomic metadata swap via epoch numbers for clients.

**Trade-offs & monitoring**- Hash + consistent hashing reduces moves but sacrifices range queries.- Monitor replication lag, hotkeys, rebalancer throughput; automate rollback on errors.

This design balances low-latency lookups with online resharding and minimal downtime suitable for a backend developer to implement on cloud databases.

MediumSystem Design

154 practiced

Design a cost-effective storage tiering strategy for application data that includes hot transactional data in PostgreSQL, semi-hot analytics data, and cold archives on object storage (S3 or equivalent). Explain partitioning, TTLs, lifecycle policies, query routes, and access patterns that justify migration between tiers while controlling egress and retrieval costs.

MediumTechnical

83 practiced

You implemented a caching layer expected to improve p99 latency. Describe an experiment and analysis plan to measure whether p99 improved in production. Include how you would collect and aggregate samples, warm-up period, time windows, calculating confidence intervals for heavy-tailed data, and how to detect false positives from noise.

Sample Answer

**Approach summary**

Run a controlled rollout (canary or A/B) and measure p99 before/after with statistical rigor and observability for confounders.

**Experiment design**

- Use A/B: route ~5–20% traffic to cached version (treatment) and rest to baseline. Keep traffic routing stable for the test.- Run a short warm-up (e.g., 5–15 minutes) after rollout to fill the cache and reach steady state; exclude warm-up from analysis.

**Data collection & aggregation**

- Collect per-request latency with timestamps, route tag (control/treatment), trace id, host, and request attributes. Export to a time-series store or analytics pipeline.- Aggregate into fixed windows (e.g., 1-min or 5-min buckets) to preserve temporal structure and detect drift. For each window compute p50, p95, p99, request count, error rate, throughput, and CPU/memory.

**Handling heavy-tailed data & CI**

- For p99 use bootstrap or quantile sketches (t-digest) to estimate CI: - If you have raw samples: use stratified block bootstrap (resample windows) to preserve time dependence and compute bootstrap distribution of p99; report 95% CI. - If using sketches: use the sketch’s quantile CI method or bootstrap on bucket summaries.- Optionally log-transform latencies for diagnostics, but compute quantiles on original scale.

**Detecting false positives / noise**

- Use A/B with hypothesis test on bootstrap distributions; require CI for difference not crossing zero and consistent improvement across time windows.- Run permutation test as a robustness check.- Monitor confounders: traffic mix, error spikes, GC/CPU, downstream latency. If confounders correlate with change, treat result as inconclusive.- Require sustained improvement over multiple time windows (e.g., >95% CI improvement for at least N consecutive windows) to avoid transient noise.

**Outcome metrics**

- Primary: absolute and relative change in p99 with 95% CI.- Secondary: p95/p50, error rate, throughput, CPU, cache hit rate.

This gives statistically grounded evidence that p99 improved and helps avoid false positives from heavy tails or operational noise.

MediumTechnical

75 practiced

Explain how you'd design an SLO-based monitoring and alerting system for backend services. Specify SLIs you would pick for a typical HTTP API, how to set SLO targets and error budgets, burn-rate alerts, ticketing/incident response flow, and what playbook steps on-call engineers should follow when an SLO breach is detected.

Sample Answer

**Overview / goals**Design an SLO-based system that protects user experience while enabling product velocity: measure SLIs for key user journeys, set SLOs and error budgets, alert on burn rate and threshold breaches, automate ticketing, and provide an on-call playbook for remediation and postmortem.

**SLIs for an HTTP API**- Availability: successful responses / total requests (2xx/3xx vs all)- Latency: p99 and p95 request latency for user-visible endpoints- Error rate: 5xx rate, and client-visible errors (4xx) for specific flows- Throughput/Capacity: requests per second vs capacity throttles

**Setting SLOs & error budgets**- Choose SLOs based on business criticality (e.g., Availability 99.9% monthly for payments; 99.5% for non-critical APIs).- Error budget = 1 - SLO (e.g., 0.1% budget per month). Convert to allowed errors: allowed_errors = total_requests * error_budget.

**Burn-rate alerts**- Compute burn rate = observed_error_rate / error_budget_rate.- Alert tiers: - Warning: burn rate > 1 for 1 hour - Urgent: burn rate > 3 for 15 minutes - Paged: burn rate > 6 for 5 minutes- Also alert on absolute SLO breach probability (e.g., projected to exhaust budget before period end).

**Ticketing & incident flow**- Integrate alerts with PagerDuty -> create incident linking SLO dashboard, recent deploys, and runbook.- Triage owner acknowledges, assigns severity, and records timeline in ticket.- If surge from deploy — roll back or disable feature flag; if infra — scale or failover.

**On-call playbook steps**1. Confirm: verify SLO dashboard, check alert scope, correlated metrics (errors, latency, traffic).2. Scope: isolate affected services/endpoints and clients.3. Mitigate: apply quick fixes (rollback, scale, throttle, circuit-breaker, disable feature).4. Root cause: gather logs, traces, recent deploys, config changes.5. Communicate: update ticket, stakeholders, and customer-facing status page.6. Post-incident: run postmortem, quantify error budget spent, action items, and update SLOs or runbooks if needed.

**Why this works**- Focuses on user impact not just infrastructure signals, ties alerts to actionable error budgets, and ensures clear on-call steps to balance reliability and speed.

EasyTechnical

98 practiced

List and explain the major cloud cost drivers for a backend-heavy SaaS product on AWS or Azure, including compute, storage, network egress, managed services (DB, caches), and operational overhead. For each driver suggest one practical optimization that reduces cost without significantly impacting latency or reliability.

Unlock Full Question Bank

Get access to hundreds of Performance Cost Optimization & Resource Efficiency interview questions and detailed answers.

Join thousands of developers preparing for their dream job.