System Design and Architecture Fundamentals Questions

Comprehensive coverage of designing scalable, reliable, and maintainable software systems, combining foundational concepts, common architectural patterns, decomposition techniques, infrastructure design, and operational considerations. Candidates should understand core principles such as horizontal and vertical scaling, caching strategies and placement, data storage trade offs between relational structured query language databases and non relational databases, application programming interface design, load distribution and fault tolerance. They should be familiar with architectural styles and patterns including client server and layered architectures, monolithic and microservices decomposition, service oriented and event driven designs, gateway and proxy patterns, and resilience patterns such as circuit breakers and asynchronous processing. Assessment includes the ability to decompose a problem into logical components and layers, define component responsibilities, map data flows between ingestion processing storage and serving layers, and select appropriate infrastructure elements such as application servers caches message queues and database replication models. Interviewers evaluate estimation of scale and load and reasoning about trade offs such as consistency versus availability and partition tolerance latency versus throughput coupling versus cohesion and cost versus complexity, and the ability to justify architecture decisions. Candidates should be able to sketch high level designs, communicate architecture to technical and non technical stakeholders, propose migration paths such as when to combine or transition between patterns, and describe operational runbooks including failure mode mitigation monitoring observability and incident recovery. Practical topics include caching eviction policies such as least recently used and least frequently used load balancing approaches such as round robin and least connections rate limiting techniques replication and sharding strategies and design choices for synchronous request response versus asynchronous queue based messaging. Emphasis is on clarifying requirements estimating constraints proposing reasonable architectures and articulating trade offs and evolution paths rather than only low level implementation details.

MediumTechnical

84 practiced

Explain LRU (least-recently-used) versus LFU (least-frequently-used) cache eviction policies. Given a product catalog service with seasonal hot items and a long tail of infrequent items, which policy is more appropriate and why? Discuss hybrid approaches and operational implications such as memory overhead and workload characteristics.

Sample Answer

**Compare LRU vs LFU (brief)**

- LRU (least-recently-used): evicts the item that hasn't been accessed for the longest time. Simple, low overhead (typically a doubly linked list + hash map), adapts quickly to changing hot sets — good when recency matters.- LFU (least-frequently-used): evicts the item with smallest access count. Good where long-term popularity matters. Requires counters (more memory) and strategy for aging to avoid keeping once-popular items forever.

**Recommendation for product catalog with seasonal hot items + long tail**

- Use a recency-first policy (LRU or LRU-dominant hybrid). Seasonality causes short-term spikes; LRU naturally favors recently hot items and will drop stale but historically popular items quickly when they stop being accessed.- Pure LFU would retain long-lived counters for previously popular seasonal items and resist adapting when trends change unless you implement decay.

**Hybrid approaches**

- Windowed LFU or LFU with decay: keep frequency but apply time decay so recently frequent items rank higher — balances long-term popularity and recency.- Multi-tier: small LRU for "recent" fast cache + LFU for "stable" popular items (example: W-TinyLFU / Caffeine library). New items go to LRU; frequent promotions to LFU tier.- TinyLFU admission filter + LRU eviction is a proven pattern: low overhead admission, high hit rate.

**Operational implications**

- Memory: LFU needs counters per key (integers or approximate sketches like Count-Min), increasing footprint. Hybrid adds structures (tiers, filters).- Complexity & latency: LRU O(1) updates; LFU naive updates can be heavier. Approximations (sketches) trade accuracy for performance.- Workload telemetry: monitor hit rate, eviction rates, item lifetime; collect time-series by key popularity to tune decay windows and tier sizes.- Failure & persistence: decide whether to persist frequency state across restarts (helps LFU) or reset on deploys (lets cache adapt).- Tuning: choose decay half-life or LRU capacity per tier to match season length and tail size.

Summary: For seasonal catalog with a long tail, prefer LRU or a hybrid (LRU front + LFU/stable tier with decay). This gives fast adaptation to trends while retaining some benefit from longer-term popularity, with acceptable memory/complexity trade-offs when using approximate counters.

MediumTechnical

69 practiced

For a social 'feed' feature at 200M users with skewed popularity, discuss when to use replication versus sharding to achieve read and write scalability. Cover fanout-on-write vs fanout-on-read trade-offs, storage cost, read latency implications, handling hot users, and partition management.

Sample Answer

**Situation & goal**I’d design a feed for 200M users with highly skewed popularity to scale reads/writes while avoiding hot partitions and excessive cost.

**Replication vs Sharding — when to use each**- Use sharding (hash or range) as primary scale mechanism to distribute write throughput and storage across many nodes. It prevents single-node write bottlenecks.- Add replication per shard for read scalability and availability. Replication is for low-latency fanout reads and failover; sharding is for write/space scaling.

**Fanout-on-write vs Fanout-on-read**- Fanout-on-write: push updates to followers’ feeds at write time. Pros: very low read latency; ideal for high read-to-write ratio. Cons: high storage and write amplification (bad with celebrity users).- Fanout-on-read: compute/feed merge on read time. Pros: lower storage and no massive write fanout for hot producers. Cons: higher read latency and CPU; requires efficient merge + caching.

Hybrid: fanout-on-write for normal users; for hot users (celebrities) keep a lightweight producer timeline and do fanout-on-read or use cached partial fanouts.

**Storage cost & read latency**- Fanout-on-write increases stored copies proportional to followers; cost explodes with skew. Estimate average followers * posts/day to project bytes.- Fanout-on-read saves storage but increases read CPU/latency; mitigate with read replicas and per-user caches (Redis, CDN) and precomputed top-K.

**Handling hot users**- Isolate hot producers into special shards or the “celebrity” tier. Apply: - Rate limiting and batching of writes - Separate fanout strategy (on-read) with prioritized caching - Use bloom filters / pointers to avoid scanning all followers

**Partition management**- Use consistent hashing with virtual nodes to rebalance without large data moves.- Monitor shard load and support dynamic resharding: split hot shards, migrate ranges, or add read replicas.- Automate alerts + runbooks for hot-partition mitigation and graceful resharding.

**Trade-offs summary**- Shard for write/space scale; replicate for read scale and HA.- Choose fanout-on-write for low read latency at storage cost; fanout-on-read for storage savings at read-cost.- Hybrid + hot-user isolation + automation gives best practical balance at 200M users.

MediumSystem Design

69 practiced

Design an asynchronous image processing pipeline: ingestion API accepts uploads, worker pool performs transformations (resize, watermark), and processed assets are served via CDN. Describe components (message queue, worker autoscaling, storage), failure handling (retries, poison queues), idempotency, backpressure handling, and how to scale workers for bursty upload traffic.

Sample Answer

**Overview / Requirements Clarification**Ingest user uploads via an API, run async transformations (resize, watermark), store processed artifacts behind CDN, support bursty traffic, ensure reliability (retries, poison queue), idempotency and backpressure.

**High-level components**- Ingestion API + auth: accepts multipart upload or presigned S3 URLs; validates and stores original to object storage (S3/GCS).- Message queue: durable queue (SQS, Pub/Sub, Kafka) holds tasks referencing object keys and transformation spec.- Worker pool: stateless workers pull tasks, download source, run transforms, upload results to storage, write metadata and CDN invalidation hooks.- Storage: object store for originals and outputs; metadata DB (DynamoDB/Postgres) for status and idempotency keys.- CDN: serves processed assets; use signed URLs and cache-control.

**Failure handling**- Retries with exponential backoff and max attempts tracked in message attributes.- On permanent failure, move message to poison queue for manual inspection and alerting.- Workers checkpoint progress and emit structured logs/metrics; use DLQ alerting + Sentry.

**Idempotency**- Use a unique task id (hash of user id + object key + transform parameters). Before processing, worker does a conditional write to metadata DB (insert if not exists). If record exists and status = complete, skip work; if in-progress beyond TTL, allow safe retry using worker lease.

**Backpressure & flow control**- API returns accepted (202) only after enqueueing; if queue depth or storage errors exceed thresholds, API returns 503 with Retry-After.- Use request rate-limiting and client-side throttling hints.- Monitor queue length and consumer lag; autoscale consumers accordingly.

**Worker autoscaling for bursts**- Use metrics-driven autoscaling: queue length, consumer lag, average processing time, and CPU/GPU usage.- Provision a baseline pool; scale up rapidly via fast-start instance types (e.g., container tasks on Fargate or k8s HPA with KEDA using queue depth).- Consider warm standby workers and pre-warmed container images to reduce cold-starts.- Cap concurrency per worker to avoid I/O saturation; use spot instances for cost with fallback to on-demand.

**Trade-offs**- SQS-style at-least-once semantics requires idempotency logic.- Kafka can provide stronger ordering but higher operational complexity.- Aggressive autoscaling speeds processing but increases cost; balance with backpressure and SLA targets.

This design yields reliable, scalable async processing with clear failure modes, idempotency safeguards, and autoscaling tuned for bursts.

HardTechnical

74 practiced

Define appropriate SLIs and SLOs for a critical public-facing authentication API. Propose three SLIs (with methods to measure them), reasonable SLO targets, alerting thresholds, on-call runbook actions per alert level, and strategies to reduce false positives while ensuring user-impacting incidents remain visible.

Sample Answer

**Context / goals**I would treat the authentication API as critical user-path: availability, correctness, and latency directly impact sign-ins, payments, and security flows. My SLIs/SLOs focus on user-observable outcomes, measurable from production ingress and client response.

**Three SLIs (measurement method)**1. **Successful authentication rate** — % of auth requests that return HTTP 2xx and valid token (measured by sampling real traffic + synthetic probes; correlate with downstream token validation logs).2. **End-to-end auth latency (p95)** — time from request arrival at edge to token issuance (measured by distributed tracing and edge latency histograms).3. **Error-rate for user-facing errors** — % requests returning >=500 or security-blocking 4xx (measured at API gateway and enriched with user-impact tags).

**SLO targets & alert thresholds**- Successful auth rate: SLO 99.95% over 30 days. Pages at 99.9% (P1), warns at 99.97% (P2).- Latency p95: SLO 300 ms. Pages if p95 > 500 ms for 5m (P1); warn if p95 > 350 ms for 10m (P2).- User-facing error-rate: SLO < 0.1% errors. Pages if > 0.5% for 5m (P1); warn if > 0.2% for 10m (P2).

**On-call runbook actions**- P2 (warning): Check synthetic probe dashboards, recent deployments, config flags; review downstream dependency metrics (DB replica lag, IdP latency); roll forward telemetry tracing spans for sample requests; if likely transient, document and monitor 30–60m.- P1 (page): Immediately acknowledge, run "isolation checklist": rollback recent deploys, toggle traffic to standby cluster, throttle non-essential flows, collect traces and request dumps, engage DB/identity vendor if dependency implicated; open incident with timeline, notify Product/Security if tokens possibly compromised.- Post-incident: RCA, SLI/SLO adjustment if needed, playbook updates, and corrective automation.

**Reducing false positives**- Use user-impacting composite SLIs (combine success + latency) rather than raw infra metrics.- Require corroboration: alert only when both edge synthetic probes and a percentage of real-user samples show failure.- Apply dynamic baselining (anomaly detection tuned to hourly patterns) and maintenance windows/deploy-exemptions.- Implement alert deduplication and a short confirmation window (example: alert only if condition holds for 5m and affects >X users) to avoid noise while keeping real incidents visible.

HardTechnical

67 practiced

Using the CAP theorem, design an architecture for a financial ledger service that requires strong consistency for balance queries but also desires high availability. Explain trade-offs and propose concrete techniques (consensus protocols like Raft/Paxos, leader-based replication, read-after-write guarantees, and caching patterns) to maximize both correctness and availability under realistic failure modes.

Sample Answer

**Clarify requirements & constraint (CAP applied)** - Strong consistency for balance queries (reads must reflect most recent committed writes). - High availability desired but in CAP, with network partitions we must choose C over A for correctness; otherwise risk incorrect balances.

**High-level architecture** - Shard ledger by account range; each shard is a Raft cluster (3–5 nodes) providing leader-based replication and linearizability. - Clients route to the shard leader for writes; reads by default go to leader for linearizable reads; optionally allow stale reads with explicit consistency flags.

**Concrete techniques** - Consensus: Raft for simplicity and strong leader semantics; use leader lease to optimize linearizable reads (ReadIndex) to avoid extra round trips. - Read-after-write: after a client writes, ensure client observes the leader ID or use a monotonic client token so subsequent reads go to same leader or include write’s commit index. - Leader-based replication: followers apply entries in order; use commit acknowledgements to durable storage (fsync) on leader before acknowledging high-value transfers. - Caching: use a read-through cache with versioned entries; cache only safely when served at leader or when cache holds a commit index >= needed index. Use short TTLs and cache invalidation via commit log notifications. - Failover: fast leader election tuning (e.g., randomized election timeout), pre-vote, and automated client retry with backoff and leader discovery.

**Trade-offs & failure modes** - Partition: we prefer consistency — isolated minority can't form leader. Availability drops for affected shard until majority restored. Mitigate by multi-region majority placement and read-only degraded mode with clear warnings. - Latency: leader-read/write route increases latency; mitigate with colocating clients near leaders, batching, and ReadIndex optimization. - Performance: Raft overhead; scale horizontally with sharding and separate hot-account handling (cache pinned leader replicas).

**Operational considerations** - Monitoring: leader stability, commit latency, election rates. - Testing: chaos engineering for partitions, disk loss. - Security: authenticated RPCs, encryption-in-transit, and strict audit of balance-changing commands.

This design prioritizes correctness (strong consistency) while maximizing availability within CAP by engineering for quick recovery, multi-region majority placement, and safe caching/read optimizations.

Unlock Full Question Bank

Get access to hundreds of System Design and Architecture Fundamentals interview questions and detailed answers.

Join thousands of developers preparing for their dream job.