Scalable System Architecture and Design Principles Questions
Architectural patterns principles and decision making for building systems that are maintainable resilient and able to scale. Coverage includes service decomposition and trade offs between microservice architectures and monoliths; layered and n tier architecture patterns; event driven design and command query responsibility segregation pattern; choosing synchronous versus asynchronous communication and its impact on correctness and latency; design principles such as loose coupling high cohesion separation of concerns and single responsibility; state management and session handling and when to favor stateless designs; application programming interface design versioning and contract management; front end and user experience considerations such as resource loading and progressive rendering; migration strategies for evolving systems and incremental refactoring; and how to weigh latency throughput reliability cost and development velocity when selecting architectures. Candidates should illustrate pattern selection with concrete examples and justify operational and developer experience implications.
HardSystem Design
61 practiced
Design a distributed Saga-based booking flow for a travel platform that must book flights, hotels, and car rentals as an atomic business operation. Choose orchestration or choreography, define the compensation actions for each step, describe how you persist saga state reliably, guarantee idempotency for retries, and outline reconciliation strategies when partial failures or network partitions occur.
Sample Answer
Requirements clarification:- Atomic business outcome: either full booking (flight+hotel+car) or logical rollback via compensations.- High throughput, multi-vendor integrations, must handle partial failures, retries, and eventual consistency.Choice: Orchestration-driven Saga- Rationale: Central coordinator (Saga Orchestrator) simplifies complex cross-service ordering, observability, retry policies, and consistent compensation sequencing—important for enterprise SLAs and auditability.High-level flow:1. Client requests BookingService -> creates Saga ID and initial state (PENDING).2. Orchestrator executes steps in order: ReserveFlight -> ReserveHotel -> ReserveCar.3. Each step calls the vendor service via async request/response and returns a result (SUCCESS/FAIL).4. On success, orchestrator advances state; on failure at any step, orchestrator triggers compensating actions for already-successful steps in reverse order and marks saga FAILED/ROLLED_BACK or PARTIALLY_RECOVERED.Compensation actions:- Flight: CancelReservation or ReleaseHold + refund initiation.- Hotel: CancelBooking or ReleaseDeposit.- Car: CancelRental or ReleaseHold.- Compensations are idempotent and may be retried until confirmed.Persisting saga state reliably:- Use an append-only Saga Store (e.g., event-sourced table in RDBMS or durable queue + DB) keyed by saga_id. Store events: Created, StepRequested, StepSucceeded, StepFailed, CompensationRequested, CompensationSucceeded.- Ensure ACID for local updates: orchestrator writes step intent and command to an outbox table in same local transaction, then a delivery worker publishes to vendor-messaging. Use transactional outbox pattern to avoid lost commands.- Replicate saga store and back up logs for audit.Idempotency guarantees:- Generate a unique saga_id and per-step command_id (UUID). Vendor APIs must accept idempotency-key header; store command_id -> outcome mapping on vendors. Orchestrator deduplicates incoming callbacks using command_id.- Implement at-most-once effect via dedup tables in each service (command_id -> result) and idempotent compensations (cancel if already canceled -> no-op but success).- Use optimistic concurrency control on saga state transitions (version numbers) to avoid double-processing.Retries and backoff:- Retry transient failures with exponential backoff and jitter. Classify errors (retryable, permanent).- For long-running holds (e.g., 24-hour flight hold), attach TTL and expiry handlers; orchestrator escalates or extends holds with vendors when needed.Reconciliation & partial failures:- Dead-letter queue for persistent failures; alerts to ops and automated reconciliation jobs that: - Scan saga-store for terminal non-final states (e.g., COMPENSATION_PENDING). - Re-fetch vendor state via their read APIs using stored reservation IDs; reconcile differences, trigger missing compensations or manual workflows.- Periodic compensator: background worker that retries pending compensations and enforces SLA deadlines.- Human-in-the-loop UI for manual resolution with full audit trail and one-click issue actions.Observability & audit:- Emit structured events for each saga transition to monitoring, tracing (distributed tracing), and an audit store. Provide dashboards showing stuck sagas, compensation rates, vendor error hotspots.Trade-offs:- Orchestration centralizes control and simplifies reconciliation, but is a single coordination point — make it stateless and horizontally scalable; persist state in durable store.- Choreography could reduce coupling but complicates failure ordering and observability, so not preferred for complex multi-step bookings requiring strict compensation order.Security and compliance:- Encrypt PII in saga store, retention policies for booking data, and ensure vendor tokens are rotated and stored in vault.Summary:Use an orchestrator with transactional outbox + durable saga event store, idempotent vendor contracts (idempotency keys), robust retry/backoff, compensations defined per resource, and reconciliation workers plus human escalation to guarantee business-atomic booking behavior with operational visibility.
EasyTechnical
69 practiced
Explain stateless versus stateful service designs. For user session handling, compare three approaches: sticky sessions at the load balancer, client-side tokens (JWT / cookies), and a centralized session store (e.g., Redis). For each approach, discuss horizontal scaling, failover behavior, security implications, and when you would favor a stateless design over stateful.
Sample Answer
Stateless vs Stateful (short): Stateless services don’t retain client-specific data between requests; each request contains all context needed. Stateful services keep session or context on the server (memory, disk, or external store).Compare three session approaches:1) Sticky sessions at the load balancer- Horizontal scaling: Simple to add app servers, but scaling is uneven—traffic tied to session-bearing nodes can create hotspots. Adding/removing nodes requires rebalancing or session migration.- Failover: If a node fails, sessions on that node are lost unless you replicate session data; user experience can be disrupted.- Security: Same as server-side sessions; sensitive data stays server-side, but longer-lived sessions increase attack surface. Load balancer must be secured to prevent session-hijack.- When to use: Small clusters, legacy apps where changing session model is costly, or when session state is large and hard to centralize.2) Client-side tokens (JWT / signed cookies)- Horizontal scaling: Excellent—true statelessness at the server; any server can validate token. Scaling is trivial.- Failover: Good—no server-side session to lose. Revocation is harder (require short token TTLs or revocation lists).- Security: Tokens must be signed and optionally encrypted; guard against token theft, replay, and ensure secure storage (HttpOnly, Secure cookies, same-site). Avoid embedding sensitive user data unless encrypted; use short TTLs and refresh flows.- When to use: APIs and microservices, high-scale systems, when low-latency and easy horizontal scaling are priorities, and where eventual revocation/refresh strategies are acceptable.3) Centralized session store (e.g., Redis)- Horizontal scaling: Servers remain stateless; scale app tier independently. Redis can be clustered for scale.- Failover: Better than sticky sessions—if app node dies, another can resume session by reading store. But session store is a single critical dependency; must be resilient (replication, persistence, backups).- Security: Session data kept server-side; secure Redis access, encryption in transit, and proper TTLs. More control for immediate session invalidation.- When to use: When sessions are mutable or large (shopping carts, multi-step flows), need immediate revocation, or when consistent server-side session management is required.When to favor stateless design:- High scalability and easy horizontal scaling are required- You want simpler failover and no per-node affinity- You have microservices or many instances behind autoscaling- You can tolerate token-based revocation complexity or design short-lived tokens with refreshWhen to accept stateful:- Sessions store large/mutable data not suited to tokens- Immediate server-side revocation and strong control required- Migration cost or legacy constraints make change impracticalTrade-offs summary:- Stateless (JWT): best for scale and resilience, pay the price in revocation and token security complexity.- Central store: balanced—scalable app tier, easier revocation, single-store operational need.- Sticky: easiest to implement but weakest for resilience and even scaling.Recommendation as Solutions Architect: Prefer stateless tokens for greenfield, high-scale APIs; use centralized store when session mutability, revocation, or large session payloads are required; use sticky only as a short-term migration or legacy compatibility measure.
MediumSystem Design
63 practiced
Design a rate-limiting strategy for a public API that supports three tiers: free, standard, and enterprise. Include how you identify clients (API keys, OAuth scopes), enforcement locations (edge, gateway, or service), token-bucket vs fixed-window choices for burst behavior, how to handle bursts for enterprise accounts, and how to communicate quota status to clients.
Sample Answer
Requirements / constraints:- Three tiers with different steady and burst capacities: Free (low steady, small burst), Standard (higher steady, moderate burst), Enterprise (high steady, large burst + SLA).- Public API: authenticate clients, defend against abuse, low-latency enforcement, scalable across regions.- Metrics and billing integration; real-time quota visibility.High-level design:- Identify clients via API keys (primary), optionally OAuth2 tokens with scopes for user-level auth. Associate key → account tier, rate limits, and metadata in a fast lookup store (Redis cluster + persistent DB).- Enforcement locations: - Edge (CDN / WAF) for coarse global limits and blacklists (cheap protection). - API Gateway (primary enforcement) for per-key quota checks and quick rejection. - Downstream services perform defensive local checks (fail-safe) and telemetry only.Rate-limiting algorithm:- Use token-bucket per client for steady rate + burst allowance (smooth tokens replenish at rate R, bucket size B). Token-bucket handles bursts gracefully and provides predictable smoothing.- For free tier: small bucket size and strict refill rate.- For standard: medium bucket and refill.- For enterprise: large bucket and higher refill; allow configurable burst window per customer.Handling enterprise bursts:- Tiered allowances: baseline token-bucket with a separate short-term credit queue backed by a debt ledger — allow temporary overdraft up to a configurable credit limit tied to SLA and billing alerts.- If overdraft used, mark events for billing / throttle gradual decay; require reconciliation if sustained.- Support scheduled burst windows (e.g., planned jobs) via customer-configurable quota increases and temporary tokens issued via admin API.Implementation details & data flow:- Gateway on request: authenticate key → lookup cached limits in Redis (local L1 cache) → attempt token consume atomically (Lua script in Redis) → allow or 429 with Retry-After.- Fallback: if Redis unavailable, use conservative local in-process limiter to avoid over-commit.- Telemetry: every decision emits metrics to monitoring pipeline (Prometheus/Kafka) for billing, alerts, and analytics.Communication to clients:- Use standard headers on each response: - X-RateLimit-Limit: total allowed in window (or refill rate) - X-RateLimit-Remaining: tokens left - X-RateLimit-Reset: seconds until full refill or next refill time - X-RateLimit-Burst: bucket size / burst capacity - Retry-After on 429- Provide a /quota endpoint and developer dashboard showing real-time usage, historical trends, and ability to request burst increases.- Webhooks/alerts when enterprise approaches 80/90% of quota or uses overdraft.Scalability & trade-offs:- Redis cluster for atomic token ops provides low latency and correctness; trade-off: operational complexity. Use consistent hashing and region-local Redis for latency; replicate counters for global limits with periodic reconciliation if needed.- Fixed-window counters are simpler but cause thundering bursts at window edges — not suitable for tiered burst handling.- Ensure privacy/security: rotate API keys, scope limits for OAuth, rate-limit per user vs per app where necessary.Why this design:- Token-bucket gives predictable smoothing and controlled bursts. Enforcement at gateway gives low-latency rejection and central control; edge filtering reduces noise. Enterprise overdraft + billing ties technical capability to commercial terms, enabling sales to offer negotiated SLAs. Clear headers and dashboard provide excellent developer experience.
MediumTechnical
50 practiced
Design an event-driven order-processing pipeline for an ecommerce platform with the sequence OrderPlaced → PaymentProcessed → InventoryReserved → ShipmentScheduled → NotificationSent. Describe the components (event producers, broker choice, consumer groups), required delivery semantics (at-least-once vs exactly-once), idempotency strategy, ordering constraints, and how you would handle long retries or compensating actions when a downstream consumer fails.
Sample Answer
Requirements clarification:- Reliable end-to-end processing of the ordered sequence: OrderPlaced → PaymentProcessed → InventoryReserved → ShipmentScheduled → NotificationSent- Scale to many concurrent orders, tolerate partial failures, preserve per-order ordering, and enable audit/compensation.High-level components:- Event producers: API Gateway / Order Service emits OrderPlaced event (including orderId, customerId, items, version, timestamp). Payment Service emits PaymentProcessed, Inventory Service emits InventoryReserved, etc.- Event router/broker: Kafka (for strong ordering and high throughput) or Amazon SNS+SQS FIFO / MSK. Use topic-per-event (order-placed, payment-processed, ...). Partition by orderId to preserve per-order ordering.- Consumer groups: Each logical service has a consumer group for its event topic; multiple instances for scaling. Each consumer processes events for assigned partitions.Delivery semantics:- Broker: at-least-once delivery (Kafka/SQS). Design consumers for idempotency to tolerate duplicates.- Where strict exactly-once is necessary (e.g., payments), use transactional outbox pattern + broker transactions (Kafka exactly-once semantics) or payment gateway idempotency tokens.Idempotency strategy:- Each consumer records processed event id (orderId + eventSequence) in a durable store or uses an idempotency table with TTL. Use optimistic updates with version checks (compare-and-swap) for state transitions.- Use event version/state machine: process event only if current order state matches expected previous state to prevent out-of-order application.Ordering constraints:- Enforce per-order ordering by partitioning events by orderId. Within a partition, consumers process sequentially. For cross-partition operations, design compensating flows; avoid global ordering.Failure handling, long retries and compensations:- Immediate retries with exponential backoff and poison-queue: failed events after N attempts go to a Dead-Letter Queue (DLQ) with metadata and failure reason.- Automated compensating actions: For irreversible downstream failure (e.g., inventory unavailable after payment), emit compensating events (PaymentRefundRequested) from an orchestrator or saga participant.- Saga pattern choices: - Choreography: services emit events and act; simpler, less central coordination, but harder to reason about complex compensations. - Orchestration: a central Saga Orchestrator (statemachine) drives steps, issues commands, handles retries, timeouts, and triggers compensations; better for complex transactional flows.- Long-running retry: backoff, human-in-the-loop workflows (ticketing) for DLQ items, and visibility via monitoring and audit logs.- Observability & safety: end-to-end tracing (correlationId), metrics, SLAs, and runbooks.Trade-offs:- Kafka gives ordering+throughput; requires consumer-careful idempotency. Exactly-once across multiple services is costly; prefer at-least-once + idempotency and sagas for compensations.
MediumTechnical
65 practiced
Compare orchestration vs choreography approaches for long-running business workflows such as loan approval that include human steps and external integrations. Design both: one using a workflow engine (Temporal/Camunda) with centralized orchestration and another using event choreography. Explain failure recovery, visibility, retriability, compensation, and testability trade-offs for each approach.
Sample Answer
Orchestration (Workflow Engine — Temporal/Camunda)Design:- Central orchestrator (Temporal/Camunda) models the loan workflow as a stateful workflow: steps like intake, credit-check (external), underwriter review (human task), decision, disbursement; each step is an activity. Human tasks surfaced as "signals" / user tasks with timeouts and reminders.- Orchestrator persists state, schedules retries/backoffs, and invokes external services via activities. Compensation steps are explicit in workflow definition.Failure recovery:- Engine provides automatic durable retries, deterministic replay, and history-based recovery. If the worker or engine restarts, workflow resumes from persisted state.Visibility:- High: single source of truth; built-in timeline, state inspection, and tracing.Retriability:- Fine-grained: configure retries per activity, exponential backoff, circuit-breakers.Compensation:- Explicit compensation handlers attached to workflow; easier to implement saga-like compensations.Testability:- High: unit-testable workflows (Temporal supports deterministic tests), integration tests on activities; easier to simulate end-to-end flows.Trade-offs:- Single control plane — simpler to reason about, but centralizes logic (can be a vendor lock-in and requires the team to model domain logic inside engine).Choreography (Event-driven, Microservices)Design:- Each domain participant (intake service, credit service, UI for underwriter, disbursement service) publishes and reacts to domain events on a message bus (Kafka/event broker). Human tasks represented by events like UnderwriterRequested → UnderwriterCompleted.- Orchestration emerges from event sequences and sagas implemented via local state machines in services.Failure recovery:- Decentralized: services are responsible for idempotency and replay via durable topics. Recovery requires replaying events or compensating manually; ordering and consumer offsets matter.Visibility:- Lower out-of-the-box: need an event log viewer or correlation IDs and distributed tracing to reconstruct the saga; harder to get a single timeline.Retriability:- Managed per-service: retries via DLQs, retry topics; more variance in retry policy across services.Compensation:- Implemented as compensating events by services; harder to ensure consistency across multiple independent teams.Testability:- Harder: requires contract tests, consumer-driven contract testing, and environment orchestration for end-to-end tests; unit tests easier per service.Trade-offs:- Highly decoupled, scalable, and better aligned with autonomous teams; but distributed complexity increases operational burden, tracing, and guaranteed end-to-end correctness.Comparison summary (key concerns)- Failure recovery: Orchestration wins for deterministic recovery; choreography requires careful idempotency and replay strategies.- Visibility: Orchestration provides centralized observability; choreography needs additional tooling (correlation IDs, event store UI).- Retriability: Orchestration offers central, consistent retry semantics; choreography gives flexibility but inconsistent policies unless standardized.- Compensation: Orchestration simplifies compensations as part of workflow; choreography spreads responsibility and requires robust saga design and compensating events.- Testability: Orchestration easier for full-flow unit and integration tests; choreography demands contract testing, staging with event streams, and more complex E2E verification.Recommendation (solutions-architect lens)- For regulated, audit-heavy, long-running workflows with many human steps and clear linear progress (loan approvals), prefer orchestration (Temporal/Camunda) for visibility, recoverability, and simplicity of compensations.- For highly decoupled systems where teams own bounded contexts and throughput/scale is paramount, prefer choreography but invest in tracing, event schemas, contract tests, and strong operational practices. Hybrid: use orchestrator to manage human-driven approval flows while letting high-volume, autonomous services communicate by events.
Unlock Full Question Bank
Get access to hundreds of Scalable System Architecture and Design Principles interview questions and detailed answers.
