System Design and Architecture Fundamentals Questions
Comprehensive coverage of designing scalable, reliable, and maintainable software systems, combining foundational concepts, common architectural patterns, decomposition techniques, infrastructure design, and operational considerations. Candidates should understand core principles such as horizontal and vertical scaling, caching strategies and placement, data storage trade offs between relational structured query language databases and non relational databases, application programming interface design, load distribution and fault tolerance. They should be familiar with architectural styles and patterns including client server and layered architectures, monolithic and microservices decomposition, service oriented and event driven designs, gateway and proxy patterns, and resilience patterns such as circuit breakers and asynchronous processing. Assessment includes the ability to decompose a problem into logical components and layers, define component responsibilities, map data flows between ingestion processing storage and serving layers, and select appropriate infrastructure elements such as application servers caches message queues and database replication models. Interviewers evaluate estimation of scale and load and reasoning about trade offs such as consistency versus availability and partition tolerance latency versus throughput coupling versus cohesion and cost versus complexity, and the ability to justify architecture decisions. Candidates should be able to sketch high level designs, communicate architecture to technical and non technical stakeholders, propose migration paths such as when to combine or transition between patterns, and describe operational runbooks including failure mode mitigation monitoring observability and incident recovery. Practical topics include caching eviction policies such as least recently used and least frequently used load balancing approaches such as round robin and least connections rate limiting techniques replication and sharding strategies and design choices for synchronous request response versus asynchronous queue based messaging. Emphasis is on clarifying requirements estimating constraints proposing reasonable architectures and articulating trade offs and evolution paths rather than only low level implementation details.
MediumSystem Design
70 practiced
As an Engineering Manager designing an API gateway for microservices, describe which responsibilities belong in the gateway versus services (authentication, authorization, routing, rate-limiting, observability, protocol translation). Explain implementation options and a migration path from direct service calls to gateway-based routing without breaking contracts.
Sample Answer
**High-level principle**Keep the gateway focused on cross-cutting, network-facing concerns; keep business logic and data ownership inside services. As an EM I balance reliability, team autonomy and operability when deciding boundaries.**Responsibility split**- Gateway: - Routing / service discovery (edge routing, path/host-based) - Authentication (validate tokens, integrate with IdP) - Rate-limiting / throttling (global and per-client policies) - Protocol translation (e.g., HTTP ⇄ gRPC, WebSocket proxy) - Request-level observability hooks (tracing headers, metrics, sampling) - Basic authorization checks for coarse-grained access (client vs service)- Services: - Fine-grained authorization (resource-based, ABAC/claims checks) - Business rules, data validation, and shaping responses - Service-level metrics and detailed traces - Enforcing quotas tied to tenant/business logic**Implementation options**- Lightweight reverse proxy (Envoy, Kong) with policy sidecars for rate-limiting and auth hooks- API management platform (Apigee, AWS API Gateway) for developer portal, keys, billing- Hybrid: edge gateway for auth/rate-limit + service mesh (Istio) for intra-cluster routing, mTLS and observability**Migration path**1. Add gateway in parallel (transparent pass-through mode) — mirror traffic for testing.2. Implement auth/rate-limit at gateway for new clients only; keep existing clients calling services directly.3. Gradually flip routing per client/route (feature flags) and enforce contracts via consumer-driven contract tests (PACT).4. Push fine-grained auth into services; keep gateway as policy enforcer for tokens.5. Deprecate direct endpoints after monitoring SLA, error rates, and completing consumer migrations. Communicate API contracts, versioning strategy, and provide SDKs/migration guides.**Trade-offs**- Gateway simplifies clients but can be single point of failure — mitigate with HA, caching, and health checks.- Avoid moving business logic into gateway to preserve team autonomy and testability.This approach balances operational needs, security, and incremental rollout while minimizing contract breakage.
MediumSystem Design
86 practiced
As an Engineering Manager, define SLIs, SLOs, and an error budget for a public search API that must return relevant results quickly. Propose SLIs (latency percentiles, availability, correctness), a 30-day SLO target, an alerting strategy when error budget is depleted, and how teams should prioritize work when budgets are exhausted.
Sample Answer
**Definition & high-level goal** For a public search API we measure relevance + speed. SLIs quantify those dimensions, SLOs set targets over a 30‑day window, and the error budget = 1 - SLO allowable failures to drive trade-offs.**Proposed SLIs (measurable)** - Latency: p50, p95, p99 of end-to-end query response time (client-observed). - Availability: % of successful HTTP 200 responses for valid queries (excluding client errors) over 30 days. - Correctness (relevance): % of queries with relevance score ≥ threshold (A/B or human-labeled gold set) — or normalized click-through/CTR quality metric for production.**30-day SLO targets (example)** - Latency: p50 < 50 ms, p95 < 300 ms, p99 < 800 ms — 99% of queries must meet these. - Availability: 99.95% successful responses. - Correctness: 98% of sampled queries meet relevance threshold.Error budget (30 days) = 1 - SLO (use the tightest SLO per product priority). Example: if availability SLO is 99.95%, budget = 0.05% of queries.**Alerting strategy when budget is depleted** - Tiered alerts: - Soft (warning): consumption > 50% — notify on-call and product manager. - Hard (critical): consumption > 90% or remaining budget < 10% — page SRE/engineering lead. - Emergency (budget exhausted): immediate incident channel, halt non-essential releases. - Alerts include root metrics, recent deploys, traffic spikes, error types, and correlation with relevance metrics.**Prioritization & runbook when budgets exhausted** - Immediately: freeze risky launches and non-urgent feature work. Shift sprint capacity to reliability (hotfixes, rollbacks, capacity and cache tuning). - Short-term: mitigate (traffic shaping, rate limits, circuit breakers, autoscale, degraded-but-safe mode for heavy queries). - Mid-term: dedicate a reliability sprint (indexing improvements, query optimization, caching, instrumentation). - Communication: notify stakeholders, adjust roadmap commitments, and document lessons with postmortem and updated SLOs if needed.**Rationale** This approach balances user experience (latency + relevance) with operational realism: precise SLIs, measurable SLOs, clear alerts and a simple, actionable error-budget-driven prioritization policy that engineering teams can operationalize.
MediumTechnical
72 practiced
As an Engineering Manager advising teams, compare blue-green, rolling, canary, and feature-flag deployment strategies. For a customer-facing API with strict SLAs, recommend an approach, outline rollout and rollback steps, and describe diagnostic actions when a deployment increases error rates.
Sample Answer
**Compare strategies (short)**- Blue‑Green: Two identical environments (blue=live, green=new). Instant switch-over; minimal runtime risk but requires double infra and careful database/migration handling.- Rolling: Gradually replace instances with new version across fleet. Lower infra cost; risk window during partial rollout; simpler for stateless services.- Canary: Deploy new version to a small subset of traffic/users, monitor metrics, then expand. Best balance of safety and realism; ideal for APIs with SLAs.- Feature flags: Decouple code deploy from feature exposure. Rapid toggle, targeted cohorts, good for experiments and fast rollback without redeploys.**Recommendation for customer‑facing API with strict SLAs**I’d use canary deployment + feature flags. Canary validates real traffic under production conditions; feature flags let us quickly disable specific behavior without rollback.**Rollout steps**1. Prechecks: automated tests, contract tests, DB migration plan, SLO alert thresholds.2. Deploy to canary subset (e.g., 1% instances or traffic). Enable feature flag for canary cohort.3. Monitor key metrics (error rate, latency p50/p95/p99, throughput, CPU/memory, downstream errors) and business metrics for a defined observation window.4. Gradually expand to 5%, 25%, 100% with automated gating based on metric thresholds.5. Remove flags or keep for gradual feature exposure as needed.**Rollback steps**- If gate fails, immediately disable feature flag for affected cohort (instant).- If flag not available, halt rollout and route traffic back to previous version (traffic-weight rollback), or switch to blue environment if using blue‑green.- Run health checks, then either retry with fixes or fully revert.**Diagnostics when errors increase**- Triage quickly: correlate deployment timestamp, error types, and affected endpoints.- Check service and infra metrics (latency, CPU, mem), logs, stack traces, and traces (distributed tracing) for root cause.- Compare canary vs baseline: if only canary affected, isolate to feature/handler; if all, consider infra/config.- Rollback/disable flag if impact on SLAs. Postmortem: RCA, corrective actions (tests, schema migrations, circuit breakers), update playbooks.
HardTechnical
72 practiced
As an Engineering Manager drafting incident response procedures, create a high-level runbook for a P1 outage affecting payment processing. Include detection, initial triage, communication plan (internal and external), mitigation steps (traffic control, fallbacks, rollback), incident commander roles, post-mortem requirements, and steps to prevent recurrence.
Sample Answer
**Summary / Purpose**Runbook for P1 outage: payment processing down or <25% success. Goal: restore payment flow, protect customers, preserve data, and communicate clearly.**Detection**- Alerts: payment success rate <25% for 5m, queue growth >X, third-party errors >Y, latency >Z.- Sources: monitoring (Prometheus), APM (NewRelic), payment gateway alerts, customer support tickets, SRE on-call pager.**Initial Triage (first 10 min)**- Incident Commander (IC) accepts pager, declares P1, starts timeline.- Gather: scope (regions, payment methods), blast radius, recent deploys/config changes, third-party status.- Quick health checks: consumer queues, DB connections, gateway auth, certificates.**Roles**- Incident Commander (IC): single decision-maker for scope, priority, comms.- Tech Lead: leads root-cause troubleshooting and mitigation steps.- SRE/Infra: network, load balancers, autoscaling.- Payments SME: gateway integration, retries, idempotency.- Comms Lead: internal + external messaging.- Support Liaison: customer impact tracking.**Communication Plan**- Internal: Create incident channel (Slack + pinned doc). 5-min status, 15-min updates.- Execs: 15-min summary with impact, estimated ETA.- External: Public status page update within 15 min: “We are investigating payment issues; no card data impact known.” Follow-up every 30–60 min until resolved. For major outages, send email to affected merchants within 2 hours.- Post-resolution: timeline and next steps within 24 hours.**Mitigation Steps (ordered)**1. Traffic control: - Throttle or pause non-critical payment flows. - Route traffic away from affected region via LB.2. Fallbacks: - Enable queued offline processing mode for accepted orders. - Offer alternative payment methods (wallets, ACH) if available.3. Retry & Idempotency: - Ensure safe retry logic; pause automated retries if causing overload.4. Rollback: - If root cause suspects recent deploy, execute canary rollback plan (feature flags or previous tag).5. Third-party isolation: - Switch to secondary gateway or degrade to read-only mode for accounts if supported.**Resolution & Recovery**- IC declares “mitigated” when payments return to normal metrics for 10–15 min.- Gradual traffic ramp with monitoring; confirm no duplicate charges.- Customer support scripts for refunds and error handling.**Post-mortem Requirements (within 72 hours)**- Owner: IC or Tech Lead to draft.- Mandatory content: timeline, root cause, contributing factors, detection/response gaps, customer impact, metric evidence.- Action items: prioritized, assigned, with due dates and verification criteria.- Review: at least one blameless post-mortem meeting with Eng, SRE, Product, Support.**Prevent Recurrence**- Short-term: hotfixes, improved alert thresholds, add circuit breakers.- Medium-term: automated failover to secondary gateway, better canary deployments, stronger contract tests for payment flows.- Long-term: capacity tests, chaos engineering for payment path, runbook drills quarterly, runbook stored in central ops repo and practiced.**KPIs / Success Criteria**- Mean time to detect (MTTD) < 5 min, MTTR < 30 min for similar incidents, no duplicate charges, post-mortem completed and 80% action items closed on schedule.This runbook balances speed, safety, and customer transparency; adapt details to team size and platform specifics.
HardSystem Design
79 practiced
As an Engineering Manager migrating notifications and reporting from synchronous REST to an event-driven architecture, outline incremental migration steps. Include dual-writing strategies, consumer contract evolution, managing data duplication, validating end-to-end correctness, and a plan to decommission old synchronous paths.
Sample Answer
**Overview / Goals**Migrate notifications & reporting from sync REST to event-driven incrementally with zero customer-impact, observable correctness, and clear rollback/decommission criteria.**Phased Migration Steps**1. Discovery & contracts - Inventory sync endpoints, producers, consumers, SLAs, payloads. - Define stable event schema and consumer-driven contract tests (Pact or similar).2. Dual-write & fan-out - Implement dual-writing at producer side: continue REST call + publish event to message bus behind a feature flag. - Use an adapter layer/service to encapsulate dual-write and retries, ensuring idempotency and at-least-once semantics.3. Consumer evolution - Start new event consumers for notifications and reporting reading from bus; initially they shadow behavior of sync flow. - Use consumer contract tests to validate event compliance continuously. - Introduce schema versioning and backward-compatible changes (additive fields, optional fields, version header).4. Data duplication & reconciliation - Accept temporary duplication: events + existing persistent state. - Build a reconciliation job (CDC or scheduled diff) to compare event-derived state vs sync-backed state; surface mismatches in dashboards. - Maintain canonical source-of-truth marker until cutover.5. Validate end-to-end correctness - Canary rollout: route a small % of live traffic to event path; run parity checks (metrics, counts, content). - Automated E2E tests simulating failures (out-of-order, retries). - Observability: business metrics, SLA latency, consumer lag, error rates, and reconciliation error trends.6. Cutover & decommission - Once parity thresholds met (e.g., 99.9% match over 48–72h, acceptable latencies), promote event path to primary for subsets, then all traffic. - Gradually disable dual-write feature flag; keep read-only sync endpoints for a grace period. - Final decommission when reconciliation shows zero/new acceptable drift and stakeholders sign off. Archive logs, remove contracts, retire infra.**Operational/Team Plan**- Cross-functional migration squad with producer, consumer, QA, and SRE.- Weekly milestones, clear rollback plans, runbooks for incidents.- Metrics-driven gating and stakeholder demos.**Risks & Mitigations**- Inconsistency: reconciliation + idempotent writes.- Contract drift: consumer-driven contract testing CI.- Latency/regression: canary + throttling.This balances engineering safety, visibility, and speed while giving teams safe rollback and clear decommission criteria.
Unlock Full Question Bank
Get access to hundreds of System Design and Architecture Fundamentals interview questions and detailed answers.
