System Design and Architecture Fundamentals Questions

Comprehensive coverage of designing scalable, reliable, and maintainable software systems, combining foundational concepts, common architectural patterns, decomposition techniques, infrastructure design, and operational considerations. Candidates should understand core principles such as horizontal and vertical scaling, caching strategies and placement, data storage trade offs between relational structured query language databases and non relational databases, application programming interface design, load distribution and fault tolerance. They should be familiar with architectural styles and patterns including client server and layered architectures, monolithic and microservices decomposition, service oriented and event driven designs, gateway and proxy patterns, and resilience patterns such as circuit breakers and asynchronous processing. Assessment includes the ability to decompose a problem into logical components and layers, define component responsibilities, map data flows between ingestion processing storage and serving layers, and select appropriate infrastructure elements such as application servers caches message queues and database replication models. Interviewers evaluate estimation of scale and load and reasoning about trade offs such as consistency versus availability and partition tolerance latency versus throughput coupling versus cohesion and cost versus complexity, and the ability to justify architecture decisions. Candidates should be able to sketch high level designs, communicate architecture to technical and non technical stakeholders, propose migration paths such as when to combine or transition between patterns, and describe operational runbooks including failure mode mitigation monitoring observability and incident recovery. Practical topics include caching eviction policies such as least recently used and least frequently used load balancing approaches such as round robin and least connections rate limiting techniques replication and sharding strategies and design choices for synchronous request response versus asynchronous queue based messaging. Emphasis is on clarifying requirements estimating constraints proposing reasonable architectures and articulating trade offs and evolution paths rather than only low level implementation details.

MediumSystem Design

78 practiced

Design an API that gracefully handles partial failures when downstream services degrade or fail. Define how you would return degraded responses to clients, implement fallbacks and circuit breakers, surface error details without leaking internals, log incidents for postmortem, and measure user impact of degraded functionality.

Sample Answer

Requirements & constraints:- API must remain available when some downstream services degrade; return best-effort responses quickly.- Preserve security (no internal stack traces), observable for postmortem, measure user impact (latency, errors, feature degradation).- Low added latency, easy-to-reason fallbacks, per-service isolation.

High-level architecture:Client → API Gateway → API Aggregation Layer (or BFF) → Downstream services (auth, catalog, recommendations, payments)Cross-cutting: Circuit Breaker Library, Fallback Strategies, Feature Flags, Telemetry/Logging, Error Masking Layer.

Core components and behavior:1. API layer: orchestrates parallel requests to downstream services with per-call timeouts and deadline propagation. Returns a composed response containing both data and a degradation metadata object:- data: best-effort fields filled- degraded: boolean- degraded_reasons: sanitized codes (e.g., "rec_service_unavailable"), user_message: friendy text ("Some recommendations are unavailable")

2. Circuit breakers: use a library (Hystrix-like or resilience4j). Per downstream service CBs with thresholds on error rate/latency. On open, short-circuit calls and trigger fallbacks immediately.

3. Fallbacks:- Cached stale data (with TTL and staleness flag)- Simpler algorithmic substitute (top-sellers instead of personalized recs)- Empty-safe defaults where acceptableFallbacks chosen per-service and documented in contract.

4. Error surface without leaks:- Return stable error codes and user-facing messages; map internal errors to codes.- Provide an incident_id (UUID) in the response when degradation occurs; clients can show generic message and include incident_id for support.- Do not expose stack traces, hostnames, or internal URLs.

5. Logging & postmortem:- Structured logs and distributed tracing (trace/span IDs). Log full internal errors and stack traces only to internal stores (ELK or Datadog) correlated by incident_id and trace id.- Emit metrics: per-service latency, error rate, fallback rate, circuit-breaker state.- Alerting: SLO-based alerts (error budget breach), spike in fallback rate, CB opens.

6. Measuring user impact:- Define SLOs/SLAs per feature (latency, availability). Track feature degradation metric: % of requests served with degraded=true and conversion impact (A/B or holdout groups).- Instrument business metrics (e.g., click-through, checkout rate) and correlate with degradation windows via tracing and attribution.- Run canary experiments: enable fallback for a small percentage and measure delta in metrics.

Scalability & trade-offs:- Parallelize downstream calls; use bulkheads (thread/connection limits) to prevent cascading failures.- Trade-offs: more aggressive fallbacks improve availability but may reduce user experience; keep fallbacks conservative and observable.- Simplicity: prefer cached or deterministic fallbacks over complex fallback logic that can introduce bugs.

Example response payload (schema):{ "data": { ... }, "degraded": true, "degraded_reasons": ["rec_service_unavailable"], "user_message": "Recommendations are temporarily unavailable", "incident_id": "abc-123", "served_from_cache": {"recommendations": true}}

This design keeps the API responsive, transparent to clients without leaking internals, enables fast automatic mitigation (circuit breakers/fallbacks), and provides the observability and metrics needed to quantify user impact and drive postmortems.

HardTechnical

83 practiced

Compare consensus protocols such as Raft and Paxos and explain their applicability. When would you use a consensus system (like Raft) for configuration or coordination services versus relying on a strongly-consistent data store? Discuss leader election, log replication, performance implications, and common failure modes to be aware of.

Sample Answer

High-level comparison- Paxos (and Multi-Paxos) is a family of protocols proven correct for reaching agreement in asynchronous networks. It’s conceptually minimal but tricky to implement and reason about.- Raft rephrases the same guarantees (safety, liveness under partial synchrony) with a pragmatic focus: understandable leader election, log replication, and membership changes. Practically, Raft implementations (etcd, Consul, HashiCorp Raft) are more approachable and widely used.

When to use consensus vs a strongly-consistent datastore- Use a consensus system (Raft/Paxos-based) for small, critical coordination state: cluster membership, leader/lock services, configuration, feature flags, service discovery, distributed locks. Consensus guarantees linearizability and uniform view across nodes and is optimized for small write/read objects.- Use a strongly-consistent datastore (e.g., single-node leadered DB, or a distributed DB with built-in transactions and sharding) when you need high-volume, large-value data storage, secondary indexes, complex queries, or higher throughput. Consensus systems are not optimized for large blobs or high write rates.

Leader election & log replication- Leader election: Raft elects a single leader via term-based elections; the leader serializes client writes. Paxos variants elect a coordinator implicitly via proposals, which can be more subtle. A stable leader reduces round-trips and simplifies reads (leader can serve linearizable reads with leases).- Log replication: Both replicate an ordered log of commands. Raft’s AppendEntries is explicit: leader appends, replicates, awaits majority acknowledgement before committing. Multi-Paxos reduces Paxos rounds by using a long-lived leader to propose values.

Performance implications- Latency: Consensus incurs at least one round-trip to a majority for writes; committed writes are ~1 RTT + disk. Reads can be optimized (leader-local reads with leases or read index) but true linearizable reads typically require contacting majority or using lease-based guarantees.- Throughput: Bottlenecked by leader and disk/IO. Adding followers helps read scalability (if reads are stale or can be served from followers with weaker guarantees) but not write scalability. Batching, pipelining, and log compaction improve throughput.- Scale: Consensus clusters are small (3–7 nodes usually). Larger quorums increase fault tolerance but add latency.

Common failure modes and mitigations- Network partitions: Minority partitions cannot make progress; majority partition continues. Ensure quorum-aware clients and avoid split-brain by careful quorum configuration.- Leader crashes/slow leader: Causes elections and temporary unavailability. Tune election timeouts, use randomized backoff to avoid livelock.- Slow followers/IO spikes: Slow or behind followers can increase leader’s commit latency if leader waits for specific replicas; use dynamic replication strategies and snapshotting to catch up.- Log growth and storage issues: Without compaction/snapshots, disks fill. Use snapshotting and log compaction.- Membership changes: Rolling membership must preserve quorum during reconfiguration (Raft provides joint consensus).- Corrupted state/disk failures: Use checksums, backups, and monitor follower health; plan for manual intervention if quorum lost.

Practical guidance- For configuration/coordination: prefer a battle-tested Raft implementation (etcd/Consul) — simple APIs, strong consistency, good operational tooling.- For high-volume application data: use a purpose-built strongly-consistent DB or partitioned system; coordinate metadata with consensus but store bulk data elsewhere.- Tune: election timeouts, heartbeat intervals, batching, disk fsync behavior per workload. Monitor leader change frequency, commit latency, and follower lag.

This trade-off: consensus gives simple, correct coordination and linearizability for critical small state at the cost of write latency, leader bottleneck, and limited cluster size. Choose based on data size, throughput needs, and operational tolerance for leader-based availability.

HardTechnical

78 practiced

Design a streaming analytics pipeline that computes near real-time aggregates (per-minute metrics) from high-volume user events with near exactly-once semantics. Discuss choices for event ingestion, ordering guarantees, deduplication, idempotent processing, checkpointing, and how an event log (e.g., Kafka) and stateful stream processors can be composed to deliver accurate metrics.

Sample Answer

Requirements & constraints:- Per-minute aggregates with low latency (sub-second ingest → <5s emit)- High throughput (millions/sec), durable storage, near exactly-once semantics- Windowed time-based aggregations, tolerant to retries, out-of-order events

High-level architecture:User events → Ingress layer (Kafka topic partitioned by event key/time) → Stream processor (e.g., Flink / Kafka Streams) with event-time processing & state store → Aggregates sink (pre-aggregated metrics in Timeseries DB / OLAP + commit log) → Dashboard/alerts

Event ingestion:- Use Kafka as durable event log; partition by customer or logical shard to balance load and provide affinity.- Include reliable producer semantics (idempotent producers, acks=all, retries) to avoid duplicates on producer side.

Ordering guarantees & event-time:- Rely on event-time processing in processor. Embed monotonic event timestamp; processors use watermarking (e.g., periodic watermarks with allowed lateness) to handle out-of-order events and close per-minute windows deterministically.- Partitioning key ensures local ordering per key; cross-partition ordering is not required for per-key aggregates.

Deduplication & idempotency:- Include a unique event id (UUID or deterministic id) per event.- Use exactly-once processing semantics offered by chosen framework: - In Flink, use checkpointed state + two-phase commit sink to achieve end-to-end exactly-once with Kafka sinks. - For custom sinks, implement idempotent writes (upserts by event key + event sequence) or use transactional writes.- Maintain a compact per-key dedup state: store last-seen event-id or a Bloom filter / time-bounded dedup window to bound state size, evicting entries beyond allowed lateness.

Stateful processing & checkpointing:- Use a stream processor with durable state backend (RocksDB) and periodic checkpoints to durable storage (e.g., HDFS/S3). Configure checkpoint interval and retention for recovery-time/throughput trade-off.- For sinks like Kafka, leverage transactions coordinated with checkpoint completion (Flink’s exactly-once sinks / KafkaProducer transactions) so offsets and sink writes commit atomically.- On failure, processor restores state from checkpoint and replays input from Kafka starting at committed offsets — ensures no double-counting when transactions are coordinated.

Putting Kafka + processors together:- Kafka provides durable replay and partitioned ordering.- Processor consumes with committed offsets managed in sync with checkpoints; when checkpoint succeeds, commit consumer offsets and commit sink transaction. On rollback, transactions abort and offsets are not committed — replay safe.- Design windows deterministically (event-time, fixed-minute tumbling windows) and emit final aggregates only after watermark passes window end + allowed lateness; optionally emit incremental updates for real-time UX.

Operational considerations & trade-offs:- Allowed-lateness vs state retention: larger lateness simplifies correctness for stragglers but increases state size.- Dedup window sizing bounds memory/false positive rate (if Bloom filters).- Monitoring: lag, checkpoint durations, sink transaction latencies, watermark health.- If strict strict exactly-once across external non-transactional sinks isn’t possible, use idempotent upserts (write aggregated minute with primary key) or store raw aggregates to a transactional DB and compute diffs.

This design combines Kafka durability, event-time windows, checkpointed state, and transactional/idempotent sinks to provide near exactly-once per-minute aggregates at scale.

EasyTechnical

79 practiced

Compare client-server and layered (n-tier) architectural styles. For a typical web application, describe component responsibilities, data and control flow, where to place cross-cutting concerns such as authentication and logging, and one advantage and one limitation of each style when evolving the system over time.

Sample Answer

Client-server vs layered (n‑tier) — concise comparison for a typical web app:

Overview- Client-server: two primary roles — client (UI/front-end) requests services, server (back-end) processes requests and returns responses. Simple, direct interaction.- Layered (n‑tier): application split into logical tiers (presentation, application/service/business logic, data access, persistence). Each layer has a clear responsibility and typically only talks to adjacent layers.

Component responsibilities- Presentation (client/browser or mobile): render UI, validate input, orchestrate user flows.- Application/business layer: implement business rules, orchestrate use-cases, coordinate transactions.- Data access/persistence: map domain to DB, manage queries, caching, transactions.- Infrastructure/services (could be separate tier): auth, logging, message queues, external integrations.

Data and control flow- Client-server: client → server endpoint → server does work (may call DB) → response → client. Control is request/response centered.- Layered: request enters presentation layer → passes to business layer → to data layer → DB; responses bubble back up. Each layer encapsulates logic and enforces contracts.

Cross-cutting concerns- Authentication/authorization: enforce at entry points — API gateway or presentation layer for coarse-grained checks; business layer for fine-grained policies. Use centralized services (Auth service / middleware) to avoid duplication.- Logging/monitoring: use centralized infrastructure (middleware, filters, sidecars, or logging framework) that can be invoked at boundaries of layers or via aspect-oriented techniques so all layers emit structured logs.- Other concerns (caching, metrics, tracing): best placed as infrastructure layer or middleware (reverse proxy, API gateway, service mesh).

Evolution: advantage and limitation- Client-server - Advantage: simple to understand and quick to build; fewer moving parts, easier to deploy for small apps. - Limitation: monolithic servers often accumulate mixed responsibilities making it harder to scale or change specific concerns independently.- Layered (n‑tier) - Advantage: better separation of concerns — easier to test, evolve, and replace a layer (e.g., swap DB or scale presentation independently). - Limitation: can introduce extra complexity and latency; strict layering may cause boilerplate and hinder cross-layer optimizations (e.g., when performance requires bypassing layers).

Practical tip: start with clear layers but keep cross-cutting services centralized (API gateway, auth service, logging pipeline) so the architecture remains maintainable as you grow.

EasyTechnical

63 practiced

Compare monolithic and microservices architectures. For each, list benefits and drawbacks across development velocity, deployment complexity, operational overhead, testing, and team ownership. Describe one scenario where moving to microservices would be premature and potentially harmful.

Sample Answer

High-level summary:- Monolith: single deployable codebase. Simple to start, easier local dev, but can become rigid at scale.- Microservices: many small services communicating over network. Enables independent ownership and scaling, but adds operational complexity.

Comparison by dimension:

Development velocity- Monolith benefits: fast initial development (shared code, single repo, simple debugging), easy refactoring across modules.- Monolith drawbacks: as size grows, teams contend for same code, long build/test cycles slow changes.- Microservices benefits: teams can move independently, adopt tech per-service, smaller codebases reduce cognitive load.- Microservices drawbacks: cross-service changes require coordination, versioning and API compatibility add friction.

Deployment complexity- Monolith benefits: single deployment pipeline, predictable rollout, simpler rollback.- Monolith drawbacks: any change requires full redeploy; risk of large blast radius.- Microservices benefits: independent deploys reduce blast radius, can do incremental rollouts per service.- Microservices drawbacks: many pipelines, orchestration (CI/CD, service discovery), complex deployment choreography.

Operational overhead- Monolith benefits: lower infra and monitoring needs initially.- Monolith drawbacks: scaling is coarse-grained; harder to optimize resource usage.- Microservices benefits: scale services independently, isolate failures.- Microservices drawbacks: higher infra cost, need for orchestration (Kubernetes), logging/tracing across services.

Testing- Monolith benefits: easier end-to-end tests in one process; local integration straightforward.- Monolith drawbacks: tests grow large, can be slow; harder to test isolated behavior if tangled.- Microservices benefits: smaller unit tests, contract testing, clearer service boundaries.- Microservices drawbacks: must invest in integration tests, contract tests, and full-system tests; flaky network-related tests.

Team ownership- Monolith benefits: easier to share knowledge; cross-cutting changes simpler.- Monolith drawbacks: unclear ownership, teams can block each other.- Microservices benefits: clear ownership per service, faster independent decision-making.- Microservices drawbacks: requires mature DevOps and API governance; risk of duplicated effort and divergent practices.

When microservices are premature and harmful:- Scenario: early-stage product with a small team and limited traffic. If you split into services too soon you incur big overhead (multiple CI/CD pipelines, infra costs, cross-service testing, and latency) without benefits. This leads to slower feature delivery, fragmented knowledge, and costly operational burden. Better start with a well-structured modular monolith, automate CI/CD, and extract services only when clear bottlenecks, independent scaling needs, or organizational boundaries justify it.

Unlock Full Question Bank

Get access to hundreds of System Design and Architecture Fundamentals interview questions and detailed answers.

Join thousands of developers preparing for their dream job.