Core principles and patterns for designing and evaluating high level and system architectures for distributed and cloud based systems. Candidates should understand high availability and redundancy, fault tolerance and graceful degradation, and how to design stateless and stateful components. They should be able to explain scalability and capacity planning strategies including horizontal and vertical scaling, partitioning and sharding, load balancing, caching and replication, and the trade offs involved. The topic covers consistency models and the trade offs between consistency, availability and partition tolerance, performance and latency optimization, reliability and durability, security for data and access control, and cost efficiency. Candidates should be able to discuss fault domains and why critical components are replicated across availability zones and regions, as well as backup, recovery and disaster recovery approaches. Common architectural patterns such as monolithic and microservice architectures, layered design, event driven and message based systems, and command query responsibility segregation are relevant. Monitoring and observability practices including metrics, logging, distributed tracing and alerting are part of assessments, together with the ability to justify architecture decisions based on functional and nonfunctional requirements, constraints, expected load and operational complexity.
EasyTechnical
24 practiced
Compare layer 4 (L4) and layer 7 (L7) load balancing approaches. Explain DNS-based load balancing, reverse-proxy L7 load balancers, and how you'd choose between them for a geo-distributed API serving both typical REST requests and long-lived streaming connections. Include TLS termination and sticky session considerations.
Sample Answer
L4 vs L7 — quick summary:- L4 (transport layer) routes by IP/port (TCP/UDP). Fast, low latency, simple health checks, opaque to HTTP semantics.- L7 (application layer) understands HTTP: routes by URL, headers, cookies, can modify requests, do auth, observability and fine-grained traffic control.DNS-based load balancing:- Uses DNS to return different IPs (round-robin, geo-DNS) or low-TTL records to steer clients to regional endpoints.- Pros: simple, scales well, works before connection (cheap global distribution).- Cons: coarse control, caching/TTL limits agility, no per-request health info, can’t inspect traffic for advanced routing or sticky behavior.Reverse-proxy L7 load balancers:- Central or regional proxies (NGINX, Envoy, HAProxy, cloud ALBs) terminate TLS, inspect headers, route per-path, implement retries, rate-limits, observability, and cookie- or header-based sticky sessions.- Support long-lived connections (WebSockets/HTTP2) but can require connection pool tuning.TLS termination:- Terminate at edge (L7) for features like WAF, routing, certificate management. For end-to-end encryption you can do TLS passthrough at L4 or re-encrypt to origin.Sticky sessions:- L4 can only do source-IP affinity (fragile with NAT). L7 can do cookie-based sticky sessions or tokens — reliable for REST but risky for long-lived streaming (locks capacity).Recommendation for geo-distributed API with REST + long-lived streams:- Use DNS-based geo-routing to send clients to nearest region (low cost, reduced latency).- Within each region use an L7 reverse proxy/edge (or cloud regional ALB) to terminate TLS, do health checks, path-based routing, auth, observability and cookie/header-based stickiness when needed.- For long-lived streaming (WebSocket/HTTP2), prefer sticky-less designs where possible (stateless backends or shared session store). If affinity is required, use L7 cookies or a connection-aware proxy with session handoff support and tune timeouts/pools.- Consider TLS termination at edge and optional re-encryption to backend if compliance requires E2E TLS.- Add health-aware DNS failover (short TTL + active health monitors) to handle regional outages.Trade-offs: L4 for raw performance and TLS passthrough; L7 for control, security, and observability. Combining DNS geo-routing + regional L7 proxies gives best balance for latency, control, and long-lived connections.
MediumSystem Design
24 practiced
Design an observability pipeline that collects metrics, logs, and traces from 10,000 microservices producing 50GB/day of logs and 10M traces/day. Discuss ingest architecture, sampling strategies, aggregation, retention policies, indexing, alert integration, and approaches to control cost while preserving debugability.
Sample Answer
Requirements & constraints:- 10,000 microservices- 50 GB/day logs, 10M traces/day- Low-latency alerting, ability to debug incidents, cost controlHigh-level architecture:- Local agent on each host/pod (e.g., Fluent Bit for logs, OpenTelemetry Collector for metrics/traces) → local buffer + batching → regional ingestion gateways (Kafka or managed pub/sub) → processing layer (stream processors / samplers) → storage backends: - Metrics → TSDB (Prometheus remote write to Cortex/Thanos or managed metrics) - Logs → cost-optimized object store (S3) + searchable index (Elasticsearch/Opensearch or Loki + index) - Traces → trace store (Jaeger/Tempo or managed tracing)Ingest and buffering:- Agents do local batching, backpressure, and TLS. Use Kafka/pub-sub to decouple producers and processors, smoothing spikes.Sampling strategies (preserve debugability):- Traces: - Hybrid sampling: 1) Always-sample traces that exceed error/latency thresholds; 2) Head-based probabilistic sampling (e.g., 0.1–1% default); 3) Tail-based sampling in stream processor: if a sampled parent indicates error/interesting behavior, promote related traces for full retention. - Client-side adaptive sampling to reduce noise from noisy services.- Logs: - Reduce volume by structured logging, log level policies, and client-side filtering (drop debug in prod by default). - Index-only important fields; store raw logs in S3 for on-demand retrieval for a longer period.- Metrics: - High-cardinality labels should be reduced at source (aggregation, relabeling). - Use rollups: 1s/10s granularity short-term, 1m/1h long-term.Aggregation, indexing & retention:- Metrics: short-term hot TSDB (retain 15–30d at high resolution), long-term aggregated store (thirty days+ aggregated to 1m/5m) in cheap backend.- Logs: Keep full indexed logs for 7–14 days. Move raw compressed logs to S3 for 90–365 days. Index only selected fields (service, level, trace_id, timestamp, error_code).- Traces: Keep full traces for 7–30 days depending on SLO-criticality; store sampled/probabilistic traces longer. Keep error traces longer (30–90d).- Use tiered storage: hot nodes (fast SSD) for recent data → cold object store for long-term.Query & alert integration:- Expose metrics to Prometheus/Grafana for alerting. Use Alertmanager for deduplication and routing.- Alerts derived from SLOs (error rate, latency percentiles), not raw logs.- Link alerts to contextual data: include trace_id, log links, recent spans in alert payload.- Implement runbook links and automated playbooks triggered by alert labels.Cost control while preserving debugability:- Enforce structured logging and log levels to avoid unbounded logs.- Relabel/rollup high-cardinality metric labels at ingest.- Use sampling (as above) and tail-based promotion to ensure error traces preserved.- Index only essential log fields; keep raw data in cheap object storage.- Use quotas, per-service budgets, and error budgets to limit noisy services. Provide developer tools to simulate costs and request exceptions.- Use serverless/managed components where it reduces ops cost; autoscale ingestion (Kafka partitions, collector replicas) to match load.- Periodically audit high-cardinality metrics/log producers and run cost-awareness dashboards.Operational considerations:- Backpressure and SLA for ingestion; circuit-break noisy producers.- Observability of the observability pipeline itself (self-monitoring).- Test sampling thresholds and validate debugability with chaos/incident drills.- Security: encrypt in transit/at-rest, RBAC for logs/traces access.Trade-offs:- Aggressive sampling/indexing reduces cost but increases risk of missing root causes — mitigated via error-based promotion and retaining raw logs in cold storage.- Managed vs self-hosted: managed reduces ops but may increase long-term cost.This design balances scalability and cost while keeping structured links between metrics, logs, and traces so SREs can quickly pivot from alerts to traces and logs during incidents.
MediumSystem Design
27 practiced
Design a sharding strategy for a multi-tenant service storing tenant-specific metadata where tenant sizes vary widely. Compare tenant-per-shard, hash-based, range-based, and consistent-hashing approaches. Discuss rebalancing, handling hot tenants, migration complexity, and what operational tooling you'd build to support migrations.
Sample Answer
Clarify requirements:- Store tenant-specific metadata, tenants vary from tiny to very large.- Goals: availability, low latency, balanced capacity, predictable failure blast radius, low migration pain, operational observability.Comparison of approaches1) Tenant-per-shard (static allocation)- What: each tenant owns one shard (DB/node/partition).- Pros: simple isolation, easy to throttle/backup per tenant, predictable performance for each tenant.- Cons: large-tenant hotspots, poor bin-packing, many tiny tenants waste resources.- Rebalancing: heavy — migrate whole tenant data between nodes.- Hot tenants: isolate to dedicated nodes; can vertically scale.- Migration complexity: straightforward but potentially heavy IO for large tenants.2) Hash-based sharding (hash(tenant_id) -> shard)- What: uniform mapping of tenants to shards.- Pros: even distribution for many similar-size tenants, simple routing.- Cons: large variance if tenant sizes differ; migrating a hot tenant requires moving its whole mapping.- Rebalancing: moving many tenants if shard count changes.- Hot tenants: still need special casing (pin to dedicated shard).3) Range-based (range on tenant id or metadata)- What: contiguous ranges of tenant IDs assigned to shards.- Pros: good for locality (if related tenants close), easier range scans.- Cons: imbalanced when tenant sizes skew; hotspots for popular ranges.- Rebalancing: split/merge ranges; moderate complexity.- Hot tenants: can split range but may affect many small tenants.4) Consistent hashing with virtual nodes (vnodes)- What: tenants map to many small hash-buckets (vnodes) distributed across physical nodes.- Pros: smooth rebalancing when nodes added/removed (only affected vnodes move), good bin-packing of variable-size tenants by assigning multiple vnodes per tenant proportional to size.- Cons: more complex routing and metadata; large tenants still require multiple vnodes and coordinated placement.- Rebalancing: low-impact, moves subset of vnodes.- Hot tenants: allocate extra vnodes, pin some vnodes to high-capacity nodes, or split tenant across nodes with clear aggregation layer.Operational considerations & tooling- Tenant sizing metadata: store tenant size, IOPS, and traffic profile in a centralized registry to drive placement decisions.- Automated balancer: controller that assigns vnodes/tenants based on capacity, utilization, and SLOs; supports dry-run and rate-limited migrations.- Migration orchestrator: transactional, chunked data copy with checksum, dual-write (or read-forwarding), cutover once sync confirmed, and automated rollback.- Canary and traffic shaping: route small percent of traffic to target shard to validate.- Monitoring & alerts: per-tenant SLOs, per-shard utilization, migration lag, error rates; anomaly detection for sudden hotspots.- Capacity planner: simulate growth and rebalance impact; recommend node additions.- Safety features: throttle migrations, maintenance windows, automatic freeze for failing migrations, and audit logs.Recommendation (SRE lens)- Use consistent hashing with vnodes plus a tenant-size-aware placement layer. This gives operational smoothness for node changes, allows proportional allocation for large tenants, and minimizes blast radius. Build strong automation: registry, balancer, orchestrator, canary routing, and rich observability to keep migrations safe and predictable.
MediumTechnical
28 practiced
Explain eventual consistency reconciliation techniques: read-repair, anti-entropy, vector clocks, last-write-wins (LWW), and CRDTs. For each technique describe typical use-cases, operational pitfalls (e.g., tombstone buildup, metadata growth), and how they affect client-visible behavior.
Sample Answer
Brief framing: eventual consistency reconciles divergent replicas without global locks. Below I summarize five common techniques, their typical use-cases, operational pitfalls, and how they surface to clients.Read-repair- What: On reads, detect stale/ divergent replicas and synchronously or asynchronously push a fix.- Use-cases: Read-heavy caches or databases where stale reads are tolerable but corrected over time (e.g., Cassandra).- Pitfalls: Amplifies read latency when synchronous; inconsistent fixes if repair logic is buggy; causes load spikes during hot reads.- Client-visible: Usually low-latency reads; occasional slower read when repair triggers; eventual convergence after repeated reads.Anti-entropy (background sync / Merkle trees)- What: Periodic background reconciliation between replicas (full or incremental) to exchange digests and sync missing data.- Use-cases: Repair large object stores, wide-area replication where continuous consistency costs too much.- Pitfalls: Network/bandwidth cost; long convergence windows; complexity of Merkle-tree maintenance; tombstone accumulation if deletes are tracked.- Client-visible: Stale reads possible until next anti-entropy run; fewer read-time latencies than read-repair.Vector clocks- What: Per-object provenance metadata (a map of node→counter) to detect causality and concurrent updates.- Use-cases: Systems needing to detect conflicts for manual/automatic resolution (Dynamo-style stores).- Pitfalls: Metadata grows with number of writers/replicas (metadata blowup); complexity presenting siblings to clients; requires merge semantics.- Client-visible: Can return multiple versions (siblings) requiring client merge; more explicit conflict exposure.Last-Write-Wins (LWW)- What: Resolve conflicts by picking value with maximal timestamp (or scalar version).- Use-cases: Simple use cases where overwrite semantics are acceptable (session stores, TTL caches).- Pitfalls: Clock skew leads to lost updates; tombstones for deletes; non-intuitive lost user data; relies on synchronized clocks or logical clocks.- Client-visible: Deterministic single version returned but risk of silent data loss if clocks disagree.CRDTs (Conflict-free Replicated Data Types)- What: Data types with mathematically guaranteed merge semantics (commutative, associative, idempotent) so replicas converge without coordination.- Use-cases: Counters, sets, sequences for collaborative editing, presence, metrics aggregation.- Pitfalls: Choice of CRDT affects semantics (e.g., OR-Set vs G-Set); metadata overhead per element (dots/causal context) can grow; delete semantics complicated (tombstones or garbage collection).- Client-visible: Intuitive merge behavior (e.g., counts sum, adds are preserved), no exposed conflicts, deterministic convergence but sometimes surprising semantics (remove-before-add edge cases).Operational notes/mitigations- Tombstones: GC windows and anti-entropy must coordinate to avoid resurrecting deleted items; tune compaction and repair windows.- Metadata growth: Prune replica IDs, use dotted version vectors or compacted causal contexts; limit writers where possible.- Observability: Monitor repair bandwidth, tombstone counts, metadata sizes, and read latency spikes; alert on abnormal growth.- Client UX: Choose technique per SLA: LWW/CRDT for transparency and simplicity; vector clocks when client-driven merge is acceptable.Summary: pick the technique by required semantics, operational capacity, and client expectations—trade-offs are always between ease of reasoning, metadata cost, and likelihood of exposing conflicts to clients.
MediumTechnical
26 practiced
Your user profile cache is serving stale data after profile updates. Design a robust cache invalidation and coherence strategy for frequently-updated user profiles. Discuss cache-aside, write-through, write-back, TTLs, invalidation via pub/sub, and trade-offs in consistency, latency, and write amplification.
Sample Answer
Requirements & constraints:- Strong read performance for user profiles (low latency)- Updates are frequent and must be reflected quickly (soft/hard consistency options)- System must remain available under partial failure- Minimize write amplification and operational complexityRecommended hybrid strategy:1. Cache-aside (primary read path)- Reads: app checks cache, on miss fetches DB and populates cache.- Writes: app writes DB first (source of truth), then invalidate cache entry.2. Invalidation via pub/sub for coherence- After DB commit, writers publish an “invalidate user:{id}” event to a reliable topic (e.g., Kafka, Redis Streams).- Cache nodes subscribe and perform local invalidation or version bump atomically.- Use at-least-once delivery: consumers dedupe by storing last-seen version/timestamp.3. Short TTL fallback- Keep a short safety TTL (e.g., 30–120s) on cached profiles to bound staleness if invalidation is missed.4. Stronger consistency option (for critical flows)- For operations that require immediately visible updates (password/email), use synchronous cache write-through: write DB then update cache with new value (or use a transactional outbox to guarantee ordering).Trade-offs:- Cache-aside + pub/sub: low read latency, eventual consistency; small window of staleness between DB commit and subscriber invalidation; scales well, moderate write amplification (invalidate messages).- Write-through: stronger read-after-write consistency, higher write latency and write amplification (every write updates cache).- Write-back: complex, risk of data loss on failure; not recommended for user profiles where DB is authoritative.- TTLs: reduce risk of permanent staleness but increase cache churn and backend load.Operational considerations:- Ensure ordering: include monotonically increasing version numbers or timestamps with events to drop out-of-order invalidations.- Monitor: metrics for stale-read rate, invalidation lag, pub/sub consumer lag; alert on rising lag.- Resilience: durable pub/sub (Kafka), consumer groups across zones, exponential backoff on consumer failures.- Testing: chaos tests simulating lost events, partitioning, and consumer restarts.This hybrid gives low-latency reads, bounded staleness, operational observability, and options to elevate consistency where needed.
Unlock Full Question Bank
Get access to hundreds of System Architecture Principles interview questions and detailed answers.
