System Architecture and Distributed Systems Questions

Assess understanding of system architecture and distributed system principles that drive technical program decisions. Topics include component decomposition data flow fault domains replication and partitioning strategies consistency and availability tradeoffs latency and throughput tradeoffs caching sharding load balancing scaling strategies capacity planning observability and failure modes. Interviewers evaluate how candidates articulate major design decisions justify tradeoffs reason about performance and cost and connect architecture choices to program scope timelines and risk.

HardTechnical

67 practiced

Design a cross-team governance and change-control process that reduces platform incidents during major upgrades while preserving developer velocity. Include the roles involved (review boards, approvers), automated CI/CD gates and policy checks, canary and progressive rollouts, release windows, rollback and emergency-changes process, and metrics to measure the effectiveness of governance.

MediumTechnical

64 practiced

You must capacity plan an HTTP API expected to reach 100,000 requests per second peak, with 95th percentile latency under 50ms and p99 under 500ms. Outline the steps you would take, required measurements, how to calculate per-instance throughput, caching layers, backend database scaling plan, network and connection considerations, and validation strategy including load testing.

Sample Answer

**Clarify requirements & constraints**- Confirm request mix (GET/POST), payload sizes, TLS, auth, burst patterns, SLAs, deploy region(s), HA zones, cost limits.

**Measurements to collect**- Average/95/99 CPU, memory, NIC utilization per request- Request size, response size, DB QPS and latency, cache hit ratio, connection setup times

**Per-instance throughput calculation**- Measure median request processing time (service time) S (ms) under realistic load.- Calculate max RPS per instance:

text

RPS_per_instance = 1000 / S

- Add safety factor (x0.6–0.7) for headroom and spikes. Example: if S=5 ms => raw 200 RPS; with 0.7 factor => 140 RPS => need ~715 instances for 100k RPS.

**Caching layers**- Edge CDN for static and cacheable responses (reduces origin RPS).- Application-level cache (Redis/Memcached) for session/state and hot lookups; aim for >90% hit rate.- HTTP reverse proxy (NGINX or Envoy) for keepalive, rate limiting, TLS termination.

**Backend DB scaling**- Read replicas + read routing; promote replicas as needed.- Partition/shard writes by tenant/key when single-node write hits limits.- Use write-back caches / async workers for non-critical writes.- Plan for IOPS, connection pools, and multi-AZ replication lag monitoring.

**Network & connections**- Keepalives and HTTP/2 or gRPC to multiplex connections.- Tune TCP backlog, ephemeral ports, and SNI/ALPN on LB.- Use regional load balancers with cross-zone sharing; autoscale frontends.

**Validation & load testing**- Create staged tests: baseline, linear ramp to peak, sudden spike, soak (hours).- Tools: k6, Gatling, Fortio; include real traffic footprint (headers, sizes).- Measure 95/99 latencies, error rates, CPU/memory, DB lag, cache hit ratio.- Iterate: identify hotspots, optimize (caching, batching, connection reuse), retest.

**Operational needs**- Autoscaling policies, SLA alerting, dashboards, chaos tests, cost estimate accounting for instances, DB, cache, and egress.

EasyTechnical

87 practiced

You are designing a resilient datacenter and must explain the difference between a fault domain and a failure domain. Provide concrete examples (rack, power feed, network switch, availability zone), describe how you would map these concepts to cloud constructs (regions, AZs, subnets), and explain how fault/failure domain choices affect fault isolation, SLA targets and testing strategy.

EasyTechnical

64 practiced

Explain the CAP theorem and its practical implications for system design. Provide concrete examples of systems that prioritize CP or AP, and explain how you would decide which tradeoff to accept for two services: an online payment ledger and a social media feed.

MediumSystem Design

51 practiced

Design a global static asset delivery solution for a SaaS product used by millions worldwide. Include CDN selection, origin design and autoscaling, TTL and invalidation strategy, cache-busting for releases, edge vs origin caching policies, and the metrics you would track to measure latency, hit ratio and cost-effectiveness.

Sample Answer

**Clarify goals & constraints**- Global scale, low latency (<100ms median), high cache hit, cost control, fast releases, immutable/static assets.

**High-level architecture**- Multi-region origins (primary in GCP/AWS regions), global CDN (multi-CDN strategy: primary Cloud CDN/CloudFront + secondary like Fastly or Akamai via traffic steering) with POPs close to users. Origins behind regional load balancers and object storage (S3/GCS) as canonical source.

**CDN selection**- Choose providers with strong POP coverage and programmable edge (compute@edge) for header logic. Use multi-CDN for resilience and traffic/cost optimization with global load balancer or DNS steering.

**Origin design & autoscaling**- Store assets in object storage (S3/GCS) for durability and egress origin; use regional read-replicas or multi-region buckets. If dynamic transformation needed, autoscale origin app servers in ASGs/Kubernetes with health probes and auto-scaling based on request queues/CPU and 95th percentile latency.

**TTL & invalidation strategy**- Default long TTLs (24h–365d) for immutable assets. Use short TTLs (5m–1h) for variant/near-static assets. Minimize purges; prefer versioned URLs. For critical hotfixes, targeted invalidation by path or surrogate-keys; batch invalidations to reduce cost.

**Cache-busting for releases**- Embed content-hash or semantic version in filenames and build pipeline (e.g., app.ab12.js). Deploy blue-green: publish new assets to bucket, update HTML references, then invalidate minimal set (HTML) with low TTL to pick up new refs.

**Edge vs origin caching policies**- Edge: aggressive caching for immutable assets, respect Cache-Control/public, use stale-while-revalidate for UX. Origin: serve Cache-Control and Surrogate-Control headers, enable conditional GET (ETag/If-Modified-Since) and allow CDN to honor surrogate headers. Use signed URLs for private assets.

**Metrics to track**- Latency: median/P95/P99 edge latency, origin latency, time to first byte (TTFB)- Cache efficiency: global and per-POP cache hit ratio, origin egress bytes, request distribution- Cost: CDN egress $, origin egress $, invalidation costs, multi-CDN cost split- Reliability: error rates (4xx/5xx per POP), cache-fill rate, invalidation success time- Release quality: % of clients serving new asset version over time

**Operational notes**- Monitor via synthetic tests from major regions, alert on degradation in P95 latency or cache hit drop >10%. Automate invalidations and rollbacks in CI/CD. Regularly analyze POP-level hit ratios to consider adding/rewriting routing rules or regional edge logic.

Unlock Full Question Bank

Get access to hundreds of System Architecture and Distributed Systems interview questions and detailed answers.

Join thousands of developers preparing for their dream job.