Performance Cost Optimization & Resource Efficiency Questions

Optimizing for the money and resources a given level of performance consumes, not just raw speed. Covers cost-per-request reasoning, right-sizing compute and memory, efficiency of resource utilization, and trading performance against spend. Emphasizes treating cost and resource efficiency as first-class performance objectives.

HardSystem Design

72 practiced

Cross-region data replication has driven up network egress costs significantly for a global service. Provide a detailed design that reduces cross-region egress while preserving read latency guarantees for regional users. Consider techniques such as read replicas, regional caches, compression, delta replication, and placement strategies.

Sample Answer

**Clarify requirements & constraints**- Target read latency per region (P99/SLA), acceptable staleness (seconds/minutes), write throughput, peak cross-region egress cost target, regulatory constraints (data residency).

**High-level approach**- Move from full synchronous cross-region replication to hybrid: regional read-local replicas + efficient async cross-region propagation + regional caches and placement-aware writes.

**Architecture**- Primary writes land in a home region; regional read replicas receive deltas async.- Components: - Change capture layer (CDC) producing compact change events (binlog/RAFT log diffs). - Delta replication pipeline: compress + batch + dedupe + sign/seq for ordering. - Regional apply workers that replay deltas to local read-only DBs. - Multi-layer cache: regional LRU in-memory cache (Redis/ElastiCache) + edge CDN for static content. - Read-routing: region-aware load balancer directs reads to local replica/cache; fallback to nearest region with fast network routes.

**Techniques to reduce egress**- Delta replication (send only diffs, not full records).- Protocol compression (snappy/zstd), batching, and protobuf binary encoding.- Interest-based filtering / subscription: only replicate partitions/keys popular in a region.- Placement strategy: geo-partition hot keys to regions (move ownership of heavy-shard to target region).- Client-side caching + TTL tuned to allowed staleness.- Read-repair and background TTL refresh to keep caches warm.

**Consistency & latency**- Serve reads from local replica/cache for low latency; document eventual consistency window.- If strong consistency needed for some keys, use synchronous multi-region write or redirect read to primary (rare).

**Operational concerns & metrics**- Track cross-region bytes, replication lag, P99 read latency, stale-read rate.- Circuit-breakers: when egress spikes, degrade non-critical regions to read-from-cache-only.- Security: TLS, signed deltas, access control.

**Trade-offs**- Reduces egress significantly at cost of increased complexity, storage for replicas, and eventual consistency for most reads. Use selective strong-consistency for critical flows.

HardTechnical

74 practiced

Your system serializes large JSON payloads, consuming significant CPU and bandwidth. Propose a migration strategy to adopt a more efficient binary protocol for internal services, including compatibility strategy, rollout plan, performance validation, and fallbacks for mixed-version deployments.

Sample Answer

**Situation & goal** Migrate internal services from JSON to an efficient binary protocol (e.g., Protocol Buffers) to reduce CPU and bandwidth while preserving compatibility and safe rollout.

**Compatibility strategy** - Define canonical schema in protobuf with explicit field numbers and use optional/nullable fields. - Maintain JSON <-> protobuf adapters at service boundaries: producers encode protobuf; consumers accept either protobuf or JSON. Use content-type or wire-level version header for negotiation. - Backward compatibility rules: never reuse field numbers, add new fields as optional, use wrapper messages for extensibility.

**Rollout plan** 1. Library & schema stabilisation: publish proto repo, codegen pipelines, linters. 2. Dual-write phase: producers emit both JSON and protobuf (or protobuf + HTTP header). Consumers keep reading JSON. 3. Dual-read phase: consumers add protobuf decoding, prefer protobuf when available. 4. Canary: enable protobuf end-to-end for 1% of traffic across dependent services; monitor. 5. Gradual ramp: increase canary to 10/50/100% per service, with automated rollback on anomalies. 6. Clean-up: after all consumers drop JSON, remove JSON code paths.

**Performance validation** - Baseline: capture CPU, p99 latency, bandwidth, and error rates on representative workloads. - Benchmarks: microbench protobuf vs JSON (serialize/deserialize), network traces for payload sizes. - End-to-end tests: load test canary path to validate latency, resource usage, and GC behavior. - SLO checks: automated gates that block rollout if CPU decreases < expected or latency/regressions exceed thresholds.

**Fallbacks & mixed-version handling** - Runtime negotiation: service reads version header; if unknown, fall back to JSON. - Circuit-breaker: if protobuf decoding fails, increment metric and fallback to JSON for that request. - Feature flags: control per-cluster or per-service toggles for quick rollback. - Observability: high-cardinality metrics for format usage, decode errors, and performance by format.

**Operational considerations** - Security: protobuf schema reviews to avoid sensitive fields; run fuzzing. - CI/CD: enforce schema changes via MR pipelines and canary automation. - Docs & runbooks: clear rollback steps and metrics to watch during incidents.

Example: enable protobuf in billing service by dual-write, canary 1% to payments service, run 48h load tests, then ramp with automated rollback if p99 latency increases >5% or decode errors >0.1%.

MediumSystem Design

105 practiced

Design a caching strategy for a read-heavy API that requires eventual consistency for user profile data. Describe where you'd place caches (client, CDN, edge, app-level, DB), cache invalidation or TTL strategies, and how you'd measure cache hit ratio and its impact on latency and cost.

Sample Answer

**Clarify requirements**- Read-heavy, user profiles, eventual consistency tolerated (stale reads acceptable for short window), scale & cost sensitivity, low read latency.

**High-level placement**- Client: short in-memory/browser cache (1–5s) to reduce duplicate rapid reads.- CDN/Edge: cache profile avatars and public metadata (e.g., display name, picture) using CDN with edge caches (TTL configurable).- App-level (API cache/Redis): authoritative read cache for assembled profile objects; regional Redis clusters for low latency.- DB: write-through/direct to primary DB; use DB as source-of-truth, no DB-level caching beyond DB buffer pool.

**TTL and invalidation**- CDN/Edge: set TTL = 30–300s for public fields; use soft purge (stale-while-revalidate) to serve while refreshing.- App cache: short-ish TTL 60–120s for eventual consistency. For profile edits, emit cache invalidation events to Redis (key delete) via message bus (Kafka/SNS). For user-sensitive changes, perform targeted invalidation + cache-warm by fetching updated profile.- Client: very short TTL + ETag-based validation to minimize wasted bandwidth.

**Consistency model**- Eventual: writes update DB, publish change event; consumers invalidate app cache and send cache-control purge to CDN asynchronously. Accept window of staleness ~ TTL + propagation.

**Measuring and impact**- Cache hit ratio: measure per layer (edge, app) via metrics: hits/(hits+misses). Instrument Redis INFO stats + CDN metrics.- Latency: track p50/p95 read latency split by cache hit vs miss; calculate latency savings = (miss latency - hit latency) * hits.- Cost: compute requests and egress saved = baseline DB reads avoided * cost per DB read / CDN egress saved. Monitor cache memory vs DB RCU costs to optimize TTL.- Alerting: low hit ratio or rising miss latency triggers TTL/invalidation investigation.

**Trade-offs**- Short TTLs reduce staleness but increase origin load; invalidation complexity increases with more layers. Use event-driven invalidation and metrics to tune.

MediumTechnical

93 practiced

Implement a small concurrent microbenchmark harness in your language of choice (Python, Go, or Java) that measures p50, p95, and p99 latency of an HTTP endpoint under a configurable concurrency level and request rate. Outline the program and key considerations (warmup, accurate timing, and avoiding client-side bottlenecks). You do not need to write full code, but describe key functions and pseudocode.

Sample Answer

**Approach (brief)**

Describe a small harness in Go (recommended for concurrency & low client overhead). Key parts: warmup, configurable concurrency and RPS, precise per-request timing, aggregator computing p50/p95/p99, and measures to avoid client-side bottlenecks (connection pooling, separate timers, CPU/network saturation checks).

**Components / Key functions**

- Config: target URL, concurrency, rps, duration, warmup- warmupPhase(duration)- worker(id, stopCh, reqCh, respCh)- rateLimiter(tokenCh) — token bucket to enforce RPS- recordLatency(timestampStart, timestampEnd) — high-resolution (time.Now())- aggregator(respCh) — collects latencies, computes quantiles at end- healthMonitor() — track client CPU, goroutine counts, open connections

**Pseudocode (Go-like)**

// spawn rateLimiter that pushes tokens at RPS
// spawn N workers
for token := range tokenCh {
  send token to reqCh
}

worker:
for range reqCh {
  start := time.Now()
  resp, err := client.Get(url) // client with Transport: MaxConnsPerHost tuned
  elapsed := time.Since(start)
  respCh <- { elapsed, success: err==nil && status<500 }
  discard body
}

**Timing & accuracy**

- Use monotonic clock (time.Since/time.Now monotonic)- Time per request inside worker immediately around the network call to avoid queue/timer skew- Avoid aggregating in hot path: send raw latency to buffered channel

**Warmup**

- Run warmupPhase that sends low-rate traffic for X seconds, discard latencies to let caches/connection pools stabilize

**Avoiding client-side bottlenecks**

- Use pooled HTTP client, reuse TCP connections, tune Dialer and Transport (MaxIdleConnsPerHost)- Ensure single-machine limits not exceeded: monitor CPU, NIC, file descriptors- Use multiple client machines if needed; measure client-side saturations and report

**Aggregation & stats**

- At end, sort latencies and compute percentiles- Report p50, p95, p99, throughput, error rate, tail latency distribution

**Trade-offs**

- Single-process harness is simple but may bias tail latencies if client saturates; distribute load for high RPS.

EasyTechnical

136 practiced

Explain basic server memory management concerns for a long-lived backend process (e.g., a caching or database process). Include swapping, memory leaks, and the impact of garbage collection pauses on latency-sensitive services.

Unlock Full Question Bank

Get access to hundreds of Performance Cost Optimization & Resource Efficiency interview questions and detailed answers.

Join thousands of developers preparing for their dream job.