System Design and Architecture Fundamentals Questions

Comprehensive coverage of designing scalable, reliable, and maintainable software systems, combining foundational concepts, common architectural patterns, decomposition techniques, infrastructure design, and operational considerations. Candidates should understand core principles such as horizontal and vertical scaling, caching strategies and placement, data storage trade offs between relational structured query language databases and non relational databases, application programming interface design, load distribution and fault tolerance. They should be familiar with architectural styles and patterns including client server and layered architectures, monolithic and microservices decomposition, service oriented and event driven designs, gateway and proxy patterns, and resilience patterns such as circuit breakers and asynchronous processing. Assessment includes the ability to decompose a problem into logical components and layers, define component responsibilities, map data flows between ingestion processing storage and serving layers, and select appropriate infrastructure elements such as application servers caches message queues and database replication models. Interviewers evaluate estimation of scale and load and reasoning about trade offs such as consistency versus availability and partition tolerance latency versus throughput coupling versus cohesion and cost versus complexity, and the ability to justify architecture decisions. Candidates should be able to sketch high level designs, communicate architecture to technical and non technical stakeholders, propose migration paths such as when to combine or transition between patterns, and describe operational runbooks including failure mode mitigation monitoring observability and incident recovery. Practical topics include caching eviction policies such as least recently used and least frequently used load balancing approaches such as round robin and least connections rate limiting techniques replication and sharding strategies and design choices for synchronous request response versus asynchronous queue based messaging. Emphasis is on clarifying requirements estimating constraints proposing reasonable architectures and articulating trade offs and evolution paths rather than only low level implementation details.

MediumTechnical

87 practiced

Design an alerting policy to reduce alert fatigue. Define alert severity tiers, SLO-derived alerts, ephemeral vs persistent alerts, automatic grouping rules, linking to runbooks, and escalation policies for on-call engineers. Explain how you would evaluate and iterate on the policy.

Sample Answer

Severity tiers- P0/Critical: customer-facing outage or SLO breach imminent — pages 24/7 to primary on-call, immediate escalation after 5 min to secondary and engineering lead.- P1/High: major degradation impacting many users — page primary with 15-min escalation, require war-room if not resolved in 30 min.- P2/Medium: degradation with workaround or limited impact — notify on-call via quiet notification, no automatic paging.- P3/Low/Informational: non-urgent telemetry, reports, or actionable items for next sprint — email/Slack only.

SLO-derived alerts- Error-budget burn alert (P1): trigger when 50% of error budget consumed in rolling window; escalate to make trade-offs explicit.- Near-SLO alert (P2): when error rate approaches threshold (e.g., 80% of budget).- SLO breach (P0): immediate page + incident activation, runbook to mitigate and start postmortem.

Ephemeral vs persistent alerts- Ephemeral (transient spikes): short window (1–5m) and require multiple samples before paging; closed automatically if metric returns to baseline — prefer alerts as non-paging notifications.- Persistent: sustained across longer window (15–30m) -> considered actionable and will page when meeting severity criteria.

Automatic grouping rules- Group by fingerprint (root cause signature), service/component, region, and deployment id to reduce duplicates.- Suppress identical alerts from all replicas; collapse flapping alerts with backoff and dedup keys.- Route grouped alerts to the owning service team channel and on-call rotation.

Runbooks and context linking- Every alert links to a concise runbook: symptom, quick checks (health endpoints, logs, common causes), mitigation steps, rollback commands, and escalation contacts.- Include playbook templates for runbook updates and a checklist to capture triage decisions.

Escalation policy- Primary on-call -> after configurable timeout escalate to secondary -> pager to engineering manager -> on-call rotation of adjacent teams if still unresolved.- Use automated status pages and post-incident notifications; require acknowledgement within SLA windows.

Evaluate and iterate- Track metrics weekly: alerts per service, MTTA, MTTR, false-positive rate, % of alerts that required paging.- Run monthly “alert hygiene” sprints to retire noisy alerts, adjust thresholds, refine grouping fingerprints, and update runbooks.- Conduct blameless postmortems for P0/P1 with action items to prevent recurrence; run A/B experiments (adjust threshold or aggregation window) and measure impact on noise vs detection.- Quarterly review with product and dev teams to align SLOs, error budgets, and alert priorities.

HardSystem Design

77 practiced

Design a distributed global rate limiter that supports bursty traffic, per-tenant quotas, and global daily quotas across regions while tolerating network partitions. Compare centralized token stores, local token caches with periodic sync, and consistent-hash approaches, and discuss correctness vs availability trade-offs.

Sample Answer

Requirements and constraints:- Functional: per-tenant rate limits (e.g., token-bucket), support bursts, global daily quotas across regions.- Non-functional: low latency (in-region), high availability during partial network partitions, eventual correctness for global quotas, multi-region scale, auditability.- Failure model: network partitions, instance failures, clock skew.

High-level architecture:- Local rate limiter sidecar per region (co-located with services) that enforces low-latency token-bucket and serves bursts.- Global quota coordinator that maintains durable daily counters and resolves cross-region accounting.- Sync layer: periodic background reconciliation + opportunistic event stream to propagate usage.

Core components:1. Local Token Bucket (per-tenant, in-memory): - Allows bursts up to token-bucket capacity. - Refill rate configured per tenant. - Fast path: grant/reject locally.2. Durable Global Store (e.g., multi-region DB or CRDT-backed store): - Stores authoritative daily usage per tenant for global quota enforcement. - Can be implemented with CRDT counters or single-writer-per-tenant shards.3. Reconciliation & Backpressure: - Local sidecars periodically push usage deltas (and on thresholds) to global store. - If global daily quota near exhaustion, global coordinator signals regions to reduce local token-bucket capacity or enter conservative mode.4. Observability & Audit: - Append-only usage log (Kafka) for reconciliation and forensic analysis.

Approach comparisons:- Centralized token store: - Model: every request checks global store. - Pros: strong correctness (single source); simple global quota enforcement. - Cons: high latency, availability depends on cross-region store; partition-prone. - Suitable when strict correctness required and traffic low.- Local token caches with periodic sync (recommended for SRE balance): - Model: serve from local buckets; asynchronously flush deltas. - Pros: low latency, tolerant to partitions (local availability); supports bursts. - Cons: temporary over-consumption risk vs global quota (bounded by bucket sizes); requires reconciliation and possible retroactive throttling. - Mitigation: conservative local caps, fast-sync on high-rate tenants, emergency global “stop” signal when quota nearly exhausted.- Consistent-hash (sharded authoritative tokens): - Model: each tenant’s counter owned by a region/replica via consistent hashing. - Pros: deterministic ownership reduces coordination; stronger correctness for owned tenants. - Cons: hot-tenant imbalance, owner failure requires re-sharding, cross-region request hops increase latency, complexity during partitions.

Correctness vs Availability trade-offs:- CAP-like trade-offs: strict global correctness (no overuse) requires synchrony and reduces availability/latency during partitions. Favoring availability means allowing bounded inconsistency (temporary overuse) with eventual reconciliation.- SRE recommendation: Prefer availability for request path (local caches) with bounded, auditable inconsistency. Set policies: - Max bounded overage = local bucket capacity. Choose size based on SLA risk. - Fast-path: local allow; slow-path: global hard-stop when daily quota reached (propagated aggressively). - For tenants requiring strict correctness, route their requests to centralized sharded owner (sacrifice latency for correctness).

Scaling and operational concerns:- Use backpressure and dynamic config push (via control plane) to reduce local capacities as global quotas deplete.- Use CRDT counters or idempotent delta writes to tolerate replays and partial failures.- Monitor reconciliation lag, quota overshoot, and emit alerts when overage exceeds thresholds.- Plan for shard rebalancing, owner failover, and clock sync (use monotonic counters, not timestamps).

Final recommendation:- Implement local token-bucket sidecars + durable global counters with idempotent delta ingestion and an aggressive push/alert path for near-quota conditions. This delivers low latency and availability while keeping global quota violations bounded and observable — a pragmatic SRE balance between correctness and availability.

HardSystem Design

76 practiced

Design a sharding and replication strategy for a social-graph service that supports low-latency neighbor reads and frequent writes (friend requests). Address shard key selection, fan-out-on-read vs fan-out-on-write, hotspot mitigation, cross-shard queries, and replication topology for availability.

Sample Answer

Requirements & constraints:- Low-latency neighbor reads (get friends/followers quickly)- High write rate (friend requests, follows)- Large skew (celebrity hotspots)- Strong availability and quick failover; acceptable eventual consistency for feeds

High-level approach:- Hybrid sharding + fanout design: store adjacency lists sharded by user-id (owner-centric) and combine with limited fan-out-on-write for light users; use fan-out-on-read for heavy users.

Shard key selection:- Primary shard key: user_id (range or consistent-hash). Use consistent hashing (virtual nodes) to simplify rebalancing and avoid large migrations.- Partition adjacency into two tables: followers_by_user and following_by_user; each row stores compressed adjacency slices (time-bucketed) to keep writes append-only.

Fan-out strategy:- For writes from user A -> user B: - If A has followers <= threshold T (e.g., 5k): fan-out-on-write — push update into followers' inbox shards (low read latency). - If A > T (celebrity): do NOT fan-out; mark as heavyweight and keep authoritative adjacency on A’s shard; reads for followers will fan-out-on-read or merge from the heavyweight store plus inboxes.- Thresholds configurable and audited by SRE.

Hotspot mitigation:- Split hotspots with virtual nodes; further horizontalize a heavy user's adjacency by sharding that user's follower list into multiple buckets (user_id + bucket_id) stored across nodes.- Rate-limit write bursts, employ backpressure and async queuing (Kafka) to smooth spikes.- Cache popular adjacency lists in read-through caches (redis cluster) with short TTL and origin validation.

Cross-shard queries:- For mutual-friends or multi-hop: route query coordinator that fans out parallel RPCs to relevant user shards and performs in-memory merge; limit to bounded parallelism and return partial results if latency SLOs require.

Replication & availability:- Use multi-AZ leader-follower per shard: - Synchronous or semi-sync replication within AZ for safety; async cross-AZ replicas for lower latency reads and fast failover. - Leader elected via consensus (Raft/Zookeeper) per shard; automatic failover with health checks. - Read replicas serve stale-tolerant neighbor reads and caches; writes go to leader.- Cross-region: active-active read-serving with CRDTs or conflict resolution for eventual consistency if multi-region writes needed.

Operational considerations (SRE):- SLOs: p99 read latency, error budget; define acceptable staleness windows for feeds.- Observability: per-shard metrics (qps, latency, replication lag), tracing for cross-shard queries, alerts on imbalance/hotspots, automated shard rebalancer.- Automation: autoscaling of storage nodes, circuit breakers for overloaded shards, blue/green deploys.- Recovery: regular backups of adjacency slices, fast restore for hot shards, runbooks for leader failover and re-sharding.

Trade-offs:- Fan-out-on-write increases write amplification but gives low-latency reads for typical users.- Fan-out-on-read reduces write cost for celebrities but increases read latency for their followers — mitigated with caches and partial precompute.- Consistent-hash + virtual nodes reduces rebalancing cost at expense of slightly uneven load; accept with autoscaling and monitoring.

EasyTechnical

84 practiced

Explain the differences between counters, gauges, histograms, and summaries in monitoring systems. Provide concrete examples (e.g., request count, CPU usage, latency distribution) and explain why histograms are often preferred for latency SLOs and how they should be aggregated across services.

Sample Answer

Counters: monotonic increasing metrics (only go up or reset). Use for totals like request_count, error_count, bytes_sent. They’re simple to aggregate across instances by summing and are ideal for rate calculations (requests/sec).

Gauges: instantaneous values that can go up and down. Use for CPU usage, current_connections, queue_length, memory_free. Gauges reflect current state and are not meaningfully summed across machines without care.

Histograms: bucketed counts + sum. Track how many observations fall into each latency bucket and the total sum. Example: request_latency_seconds_bucket{le="0.1"} = 1200, ... and request_latency_seconds_sum = 345.2. Histograms let you compute quantiles (via histogram_quantile in Prometheus), percentiles, and error budgets. They are additive across instances (you can sum bucket counts and sums) so you can aggregate across services or replicas and compute global SLOs accurately.

Summaries: client-side or per-instance quantile estimates (e.g., 0.95 quantile) plus count and sum. Useful for per-instance latency percentiles but are not safely aggregatable across instances because quantile algorithms are local—summing p95 from each pod doesn't give cluster p95.

Why histograms for latency SLOs:- You need accurate global percentile estimates for SLOs and to compute error budgets. Histograms carry the full distribution (in buckets) and are additive, enabling correct aggregation across services/instances.- They allow flexible post-hoc queries (different percentiles) without instrumenting for each quantile.

How to aggregate across services:- Use histogram buckets with consistent bucket boundaries across all services of the same SLO class.- In Prometheus, sum over instances: sum(rate(request_latency_seconds_bucket[5m])) by (le) yields aggregated bucket rates; then apply histogram_quantile(0.99, ...) to compute global percentiles.- Also sum the *_count and *_sum for average calculations.- Choose bucket boundaries to cover SLO thresholds (finer near target latencies).- Watch cardinality and memory: balance bucket resolution vs. cost; consider exemplars for tracing or using lower-resolution histograms plus sampling.

Practical tips:- Standardize bucket schemas org-wide.- Use client libraries’ histogram types (Prometheus) and expose both buckets and sum/count.- Use summaries for per-instance debugging, histograms for SLOs and global aggregation.

EasyTechnical

77 practiced

Contrast synchronous request-response architectures with asynchronous queue-based messaging for inter-service communication. Explain decision criteria an SRE or architect would use (latency vs throughput, reliability, backpressure), and discuss how to design retries, dead-letter handling, and monitoring in the asynchronous case.

Sample Answer

Synchronous request-response vs asynchronous queue-based messaging

Synchronous (HTTP/gRPC):- Pattern: caller waits for immediate response.- Strengths: low tail-latency for single requests, straightforward error semantics, easy tracing of request path.- Weaknesses: coupling between services, poor resilience under load (blocking threads), limited throughput when many callers block.

Asynchronous (message queue / pub-sub):- Pattern: producer enqueues message; consumer processes later.- Strengths: high throughput, natural load leveling, decoupling, durable buffering for resilience.- Weaknesses: higher end-to-end latency, harder causal tracing, eventual consistency.

Decision criteria (SRE/architect lens):- Latency vs throughput: choose sync when strict end-to-end latency/SLOs require immediate response; choose async for high throughput and smoothing traffic spikes.- Reliability: async with durable queues increases resilience to downstream failures; sync requires retries and circuit breakers.- Backpressure: queues absorb bursts; in sync models implement rate limiting, circuit breakers, and autoscaling.- Operational complexity: async adds operational burden (queue management, DLQs) and observability challenges.

Designing retries, dead-letter handling, and monitoring (async):- Retries: use exponential backoff with jitter, bounded max attempts, and idempotency keys so retries are safe. Prefer consumer-side retries with visibility; consider broker-delayed retries or separate retry topics to avoid head-of-line blocking.- Dead-letter queues (DLQ): route messages that exhaust retries to DLQ with metadata (error, attempt count, timestamps). Ensure DLQ retention and tooling for reprocessing, alerting, and manual inspection. Separate poison-message detection from transient-failure retries.- Monitoring & alerting: instrument: queue depth, consumer lag, processing rate, success/failure rates, retry rate, DLQ growth, and processing latency percentiles. Alert on rising lag, sustained DLQ spikes, or consumer liveness failures. Correlate messages with traces/IDs for RCA.- Observability: attach trace IDs to messages, emit structured logs and metrics per message type, and build dashboards showing end-to-end flow (producer → queue → consumer).- Capacity & SLOs: set SLOs for processing latency and availability, provision queue storage and consumer autoscaling thresholds, run chaos tests for broker outages.

These practices balance throughput and reliability while keeping end-to-end observability and operational control.

Unlock Full Question Bank

Get access to hundreds of System Design and Architecture Fundamentals interview questions and detailed answers.

Join thousands of developers preparing for their dream job.