System Design and Architecture Fundamentals Questions

Comprehensive coverage of designing scalable, reliable, and maintainable software systems, combining foundational concepts, common architectural patterns, decomposition techniques, infrastructure design, and operational considerations. Candidates should understand core principles such as horizontal and vertical scaling, caching strategies and placement, data storage trade offs between relational structured query language databases and non relational databases, application programming interface design, load distribution and fault tolerance. They should be familiar with architectural styles and patterns including client server and layered architectures, monolithic and microservices decomposition, service oriented and event driven designs, gateway and proxy patterns, and resilience patterns such as circuit breakers and asynchronous processing. Assessment includes the ability to decompose a problem into logical components and layers, define component responsibilities, map data flows between ingestion processing storage and serving layers, and select appropriate infrastructure elements such as application servers caches message queues and database replication models. Interviewers evaluate estimation of scale and load and reasoning about trade offs such as consistency versus availability and partition tolerance latency versus throughput coupling versus cohesion and cost versus complexity, and the ability to justify architecture decisions. Candidates should be able to sketch high level designs, communicate architecture to technical and non technical stakeholders, propose migration paths such as when to combine or transition between patterns, and describe operational runbooks including failure mode mitigation monitoring observability and incident recovery. Practical topics include caching eviction policies such as least recently used and least frequently used load balancing approaches such as round robin and least connections rate limiting techniques replication and sharding strategies and design choices for synchronous request response versus asynchronous queue based messaging. Emphasis is on clarifying requirements estimating constraints proposing reasonable architectures and articulating trade offs and evolution paths rather than only low level implementation details.

MediumTechnical

71 practiced

Explain the CAP theorem and apply it to a decision between Cassandra and PostgreSQL for the messaging component of a social network. Discuss trade-offs related to partition tolerance, availability, consistency, operational complexity, and developer experience when choosing either database for message delivery and persistence.

Sample Answer

CAP theorem states a distributed system can simultaneously provide at most two of: Consistency (all nodes see same data at once), Availability (every request gets a response), and Partition tolerance (system continues despite network partitions). In practice you must tolerate partitions (P) in distributed networks, so you choose a trade-off between C and A.

Applying this to messaging in a social network:

Cassandra (AP-leaning):- Partition tolerance: very strong — masterless architecture handles network splits.- Availability: excellent — writes served by any replica; low-latency write path.- Consistency: tunable per-request (ONE, QUORUM, ALL) but default favors eventual consistency; reads may see stale messages unless stricter consistency is used.- Operational complexity: requires careful data modeling, repair operations, compaction tuning, monitoring of hinted handoff and GC.- Developer experience: requires thinking in terms of denormalized wide rows and time-series patterns; less flexible for ad-hoc queries.

PostgreSQL (CP-leaning when using synchronous replication):- Partition tolerance: limited — network partitions can force failover or block synchronous commits.- Availability: high for single-node deployments; distributed availability depends on replication setup (async replication improves availability but sacrifices consistency).- Consistency: strong ACID guarantees on primary; immediate read-after-write consistency if reading from primary.- Operational complexity: mature tooling, simpler transactional semantics, easier schema evolution, but scaling writes horizontally is harder (need sharding or Citus).- Developer experience: rich SQL, transactional semantics, JSON support, easier joins and ad-hoc queries.

Decision guidance:- If user-facing message delivery must be low-latency, always-writable across geo regions, and temporary inconsistency is acceptable (e.g., feed propagation, eventual delivery), Cassandra is a better fit.- If strict ordering, no lost messages, strong transactional guarantees, and simpler developer workflow matter (e.g., direct message inbox requiring exact delivery and delete semantics), PostgreSQL (or Postgres with logical/sharded cluster or using a primary regional DB with durable queues) is preferable.- Hybrid approach: use Cassandra for global fan-out and read-heavy timelines, and Postgres for critical transactional stores (DMs, payment-related messages) — combine with a message queue (Kafka/RabbitMQ) to decouple ingestion, ensure durability, and implement exactly-once or idempotent consumers.

Ultimately balance expected partition scenarios (multi-region vs single-region), SLA for availability vs consistency, team expertise, and operational budget when choosing.

MediumTechnical

60 practiced

How would you design observability and alerting to detect and diagnose cascading failures in a microservices architecture? List the key metrics, tracing and logging approaches, SLO-based alerting rules, and immediate mitigation steps to contain cascading impact before full recovery procedures kick in.

Sample Answer

Clarify constraints: assume many small services, API/gRPC comms, autoscaling, centralized observability (Prometheus, OpenTelemetry, ELK/Tempo), and SLAs/SLOs defined per customer needs.

Key metrics (per-service and infra):- Latency p50/p95/p99, error rate (4xx/5xx), request throughput, concurrent requests, queue sizes, thread-pool / connection-pool utilization, downstream call latency/error, CPU/Memory, GC/pod restarts, circuit-breaker/circuit-open counts, retry/backoff counts, rate-limit drops.

Tracing & logging:- Distributed tracing (OpenTelemetry) with consistent trace/span IDs, capture service-to-service calls, payload sizes, and upstream/downstream annotations (circuit state, retries). Sample all errors and high-latency traces; tail-sample regular traffic.- Structured logs (JSON) with trace_id, span_id, user/request id, service version, environment; ship to centralized store (Elastic/Datadog) with retention/indices by severity.- Correlate traces+logs+metrics in dashboard; instrument health checks and readiness probes.

SLO-based alerting rules:- Page-level (P1): SLO burn-rate > 4x for error rate or latency p99 over SLO for >5min; circuit-open > threshold; high retry/backoff cascade signals.- Prod-level (P2): sustained SLO burn-rate >2x for 15+ min, elevated 95th latency or queue growth.- Noisy/Info: single-request spikes, infra warnings.- Alerting: use burn-rate, absolute thresholds, and anomaly detection; route to on-call with runbooks and include recent traces/log links.

Immediate mitigation playbook (containment):1. Stop the bleed: enable or increase network-level rate-limits and global ingress throttling (API gateway) to protect downstream services.2. Short-circuit: open circuit breakers or disable non-essential downstream calls (feature flags) to prevent propagation.3. Scale targeted services (vertical/horizontal) where safe; prefer horizontal with controlled ramp to avoid thundering herd.4. Reduce client retries and increase client-side backoff globally.5. Rollback suspect deployments or isolate problematic version via traffic-splitting.6. Engage throttling at message queues (pause consumers) if backlog growth threatens persistence layer.7. Promote read-only or degraded mode for dependent services (serve cached results).8. Triage: collect high-priority traces and error logs, map dependency graph to identify root service, and apply fixes.

Recovery & follow-up:- After containment, perform root cause analysis, adjust SLOs/alerts, add better circuit-breaker thresholds, improved autoscaling policies, and create targeted synthetic tests to catch it earlier. Document the incident and update the runbook.

This design balances detection, fast containment, and data-rich diagnosis to stop cascading failures quickly while providing traceable steps for full recovery.

HardTechnical

84 practiced

Design an asynchronous processing pipeline that deduplicates events sent multiple times by clients (at-least-once delivery). Address idempotency keys, dedupe store architecture (size, TTL, eviction), use of transactional outbox or exactly-once sinks, and how deduplication affects throughput and latency.

Sample Answer

Requirements clarification:- Clients may retry (at-least-once). We must guarantee each logical event processed once (or at least idempotent outcome), preserve ordering per key when required, scale to millions/sec, bounded storage for dedupe state, and acceptable latency.

High-level approach:- Clients supply an idempotency key (client_id + event_id or UUID). Pipeline: Ingest → Deduplication layer (fast lookup + write) → Processing → Transactional outbox → Sink (consumer/DB).- Deduplication is done at ingest boundary (edge or ingestion service) to avoid wasted work.

Dedupe store architecture:- Use a highly-available KV store (Redis Cluster or RocksDB-backed Kafka Streams state store). Keys = idempotency_key; value = {status, timestamp, result_ref}.- Size: capacity = expected QPS * dedupe_window_seconds * safety_factor. Plan sharding by client or hash ranges.- TTL: Set to dedupe_window based on business SLA (e.g., 24h). TTL removes old keys automatically to bound storage.- Eviction: LRU only as last resort — prefer explicit TTL and background compaction. If eviction unavoidable, expose metrics and degrade to “possible duplicates” mode.- Persist recent results (or pointer) so repeat requests can return cached response without reprocessing.

Consistency and correctness:- For synchronous responses, perform atomic check-and-set: GET key; if absent, write key with status=processing and proceed; on completion write status=done + result_ref. Use server-side atomic ops (Redis SETNX with expiry) or transactional state store.- For async pipelines use Kafka with transactional outbox: processing writes result + outbox event in same DB transaction; outbox consumer publishes to Kafka exactly-once (Kafka transactions) so downstream sees one write. Alternatively, use Kafka Streams with built-in state store and EOS (exactly-once semantics) to dedupe and emit once.- For cross-service dedupe, centralize or use distributed consistent hashing to route same idempotency key to same partition/instance to serialize handling.

Transactional outbox / exactly-once sinks:- Use transactional outbox pattern: app DB transaction writes business change + outbox row; separate reliable publisher reads outbox and publishes to Kafka, then marks outbox row processed. Combine with Kafka transactions for end-to-end exactly-once to downstream sinks (DBs that support idempotent writes or upserts).- Where sink supports idempotent writes (e.g., upsert by idempotency key), simpler: emit events and let sink dedupe by key.

Performance trade-offs:- Latency: Dedup check adds one lookup/write before processing. Using in-memory store (Redis) keeps extra latency ~1–5ms; local partitioned state store (embedded RocksDB in stream processors) can be sub-ms for partition-local keys.- Throughput: Stronger consistency (global locks or central store) reduces parallelism. Partitioning by key and colocating dedupe state preserves throughput linearly with partitions. Enabling persistence (durable KV) increases write IOPS—provision accordingly.- Exactly-once semantics incur overhead: Kafka EOS adds CPU and network overhead; transactional outbox adds DB transaction cost. Choose at-rest durability vs throughput trade-off per use case.

Failure modes and mitigations:- Crash between marking processing and completion -> use status=processing with lease/heartbeat + reaper to allow retry after timeout.- Clock skew -> use server timestamps and monotonic counters when needed.- Dedupe window too short -> duplicate accepted after TTL; tune TTL per business risk.

Operational considerations:- Autoscale dedupe store and partition counts based on metrics: key cardinality, write/read QPS, hit ratio.- Expose observability: dedupe hit/miss rate, eviction count, processing latency percentiles.- Provide client guidance: idempotency key format, TTL expectations, and retry backoff.

Summary:- Use client-provided idempotency keys, route keys to partition-local dedupe state, perform atomic check-and-set, persist result refs, and employ transactional outbox + Kafka transactions for end-to-end once semantics. Balance TTL and storage to bound capacity, shard to scale throughput, and accept small latency overhead for correctness.

MediumSystem Design

77 practiced

Design a URL shortening service supporting 100M stored short URLs and 1B redirects per day. Include URL generation strategy (collision avoidance), storage model, redirection latency targets, CDN and cache placement, analytics for clicks, and how to mitigate hot-key problems (very popular short URLs).

Sample Answer

Requirements & constraints:- Store 100M short URLs, handle 1B redirects/day (~11.6k reqs/s average, peaks say 100k rps).- Low redirect latency (<50ms tail), high availability, analytics on clicks, avoid collisions, handle hot keys.

High-level architecture:User -> CDN (edge caching) -> Edge cache (Redis/Local L1) -> Redirect service (stateless) -> Storage (primary DB) + write-ahead analytics pipeline -> Click processing (streaming -> OLAP).

URL generation (collision avoidance):- Use an auto-increment numeric ID (strong uniqueness) or a time-ordered KSUID/ULID if distributed. Encode ID into compact Base62 string for short path.- For distributed writers, allocate ID blocks via a central ID service (Snowflake-style) to avoid collisions.- Validate incoming custom aliases with uniqueness check + conditional write (compare-and-set).

Storage model:- Primary store: relational DB (e.g., CockroachDB / Spanner) or sharded NoSQL (Cassandra) keyed by short-id -> {long_url, created_at, owner, ttl, metadata}.- Secondary: write-through cache (Redis cluster) for hot lookups; TTL-based eviction.

CDN & Cache placement:- Put CDN in front to serve redirects for popular links (cache 301 responses). Cache key: short-id; TTL configurable by owner.- Edge cache (Redis/Envoy) for regional hot keys. Local in-process LRU for micro-latency.- Cache fill: on miss, redirect service reads DB and populates cache.

Redirection latency targets:- Aim for <20ms median, <50ms 99th; achieve via CDN edge cache, local caches, and prefetched data for known hot links.

Analytics pipeline:- Emit click events asynchronously to a durable queue (Kafka). Use stream processing (Flink) to aggregate real-time metrics and write to OLAP store (ClickHouse / BigQuery) and nearline dashboards.- Keep sampling for high-volume links to reduce load; store full logs in blob store (S3) for reprocessing.

Hot-key mitigation:- Detect hot keys via rate counters in Redis; when threshold exceeded: - Serve directly from CDN/edge cache (increase TTL). - Use read-replicas/sharded caches to spread load. - Rate-limit or tiered serving for abusive spikes (backpressure). - Pre-warm caches: promote popular short-ids to edge caches proactively based on analytics. - Use multiple routing shards for extremely popular redirects and consistent hashing to spread reads.

Trade-offs & availability:- Using numeric IDs + Base62 is simple, compact, deterministic. Snowflake-style IDs add complexity but enable horizontal write scale.- Strong consistency not required for redirects; eventual consistency acceptable for analytics.- Prioritize caching and CDN to meet latency at scale; ensure DB has sufficient replica/partitioning for write durability.

Monitoring & SLOs:- Track 99th percentile latency, cache hit ratio, error rate, and analytics freshness. Set auto-scaling rules for redirect workers, Redis shards, and Kafka throughput.

EasyTechnical

78 practiced

Explain the circuit breaker pattern and provide a concrete example of how to implement it at a service client level, including the three states (closed, open, half-open), error thresholds, cooldown/backoff behavior, and how to surface circuit state for monitoring and alerts.

Sample Answer

The circuit breaker pattern prevents cascading failures by stopping requests to an unhealthy downstream service and allowing controlled retries. It has three states:

- Closed: requests flow normally; failures are counted.- Open: requests fail-fast; no calls to downstream for a cooldown period.- Half-open: allow a small number of trial requests to test recovery; success => Closed, failure => Open.

Concrete client-level implementation (pseudo-code):

python

class CircuitBreaker:
    def __init__(self, failure_threshold=5, cooldown_seconds=30, half_open_max=2):
        self.failures = 0
        self.state = "CLOSED"
        self.open_until = None
        self.failure_threshold = failure_threshold
        self.cooldown = cooldown_seconds
        self.half_open_allow = half_open_max
        self.half_open_count = 0

    def call(self, fn, *args, **kwargs):
        now = time.time()
        if self.state == "OPEN":
            if now >= self.open_until:
                self.state = "HALF_OPEN"
                self.half_open_count = 0
            else:
                raise CircuitOpenError()
        try:
            result = fn(*args, **kwargs)
            self._on_success()
            return result
        except Exception:
            self._on_failure()
            raise

    def _on_success(self):
        if self.state == "HALF_OPEN":
            self.state = "CLOSED"
            self.failures = 0
        else:
            self.failures = 0

    def _on_failure(self):
        self.failures += 1
        if self.state == "HALF_OPEN" or self.failures >= self.failure_threshold:
            self.state = "OPEN"
            self.open_until = time.time() + self.cooldown

Key parameters and behaviors:- Error thresholds: failure count over time window (use sliding window or exponential decay) to avoid noisy spikes.- Cooldown/backoff: fixed cooldown or exponential backoff (increase cooldown after repeated open cycles).- Half-open probing: limit concurrent trial requests, use success ratio (e.g., need 3/5 successes) before closing.

Monitoring & alerts:- Emit metrics: state transitions, failures, success rate, latency, open_count, time_in_state to a metrics system (Prometheus/Grafana).- Create alerts: circuit_opened for >N opens per minute; downstream_error_rate above threshold; increased latency.- Expose health endpoints or distributed tracing tags to correlate requests blocked by circuit breaker.- Log state transitions with context (service, instance, timestamp) for root-cause analysis.

Deployment note: implement at client library or sidecar level for consistency across services; configure thresholds per downstream SLA and traffic profile.

Unlock Full Question Bank

Get access to hundreds of System Design and Architecture Fundamentals interview questions and detailed answers.

Join thousands of developers preparing for their dream job.