Design Pattern Application and Trade-offs Questions

Exploration of software design patterns and their practical application within large-scale systems, focusing on how to select appropriate architectural patterns and evaluate trade-offs between approaches (e.g., monolith vs distributed, layering vs microservices, event-driven vs request-driven, caching strategies, data consistency models, and resilience patterns) to balance scalability, maintainability, and performance.

MediumSystem Design

56 practiced

You need to implement a saga pattern for an order-processing workflow that spans inventory, payment, and shipping services. Describe whether you would use orchestration or choreography, the compensation actions for failures, how to handle partial failures and retries, and how to ensure observability across the saga.

Sample Answer

Requirements & constraints:- Workflow: Reserve inventory → Charge payment → Create shipment- Need strong failure handling, traceability, and business-visible state- Services are autonomous, event-driven, and may be unreliable/slow

Design choice: orchestration (recommended)- Why: central coordinator (Saga Orchestrator) simplifies complex roll-forward/compensate logic, enforces correct ordering (e.g., don't ship before payment), centralizes retries and visibility. Use choreography only for very simple, loosely-coupled flows where eventual consistency and no complex compensation ordering are acceptable.

High-level architecture:- API Gateway → Order Service creates saga instance in Saga Store (durable DB) and publishes StartOrder command to Orchestrator- Orchestrator (statemachine) issues commands/events to Inventory, Payment, Shipping via message broker (Kafka/RabbitMQ)- Each service responds with success/failure events; Orchestrator updates saga state and emits next command or compensation- Use Outbox pattern in services to ensure atomic write + publish

Compensation actions (examples & ordering):- If payment fails after inventory reserved: compensate by releasing inventory (InventoryService.ReleaseReservation)- If shipment creation fails after payment: issue RefundPayment and ReleaseInventory- If shipment created but later fails (carrier cancel): depending on business, either refund + cancel shipment or attempt rebook + notify customer- Ensure compensations are idempotent and reversible where possible

Handling partial failures & retries:- Idempotency tokens for every command to allow safe retries- Exponential backoff + limited retries per action. On repeated failures, escalate to manual intervention with saga state = FAILED/REQUIRES_MANUAL.- Use durable saga store to persist attempts, timestamps, and failure reasons- For transient errors: retry at service boundary. For business constraint failures (e.g., out-of-stock), perform compensation immediately.- Use circuit breaker to avoid thrashing downstream services- For long-running operations, use heartbeats/TTL and a watchdog that triggers compensation if a step is stuck beyond threshold

Observability & operational controls:- Correlation ID per saga propagated in all messages and logs- Distributed tracing (OpenTelemetry) for end-to-end traces across services- Structured logs with saga_id, step, attempt, reason- Saga Store exposes current state, history (audit log) and timeline for each saga- Dashboards/metrics: sagas started/completed/failed, latency per step, retry counts, compensation counts- Alerting on rising failure rate, long-running sagas, or manual intervention queue size- Expose a read API for operational tooling to retry/compensate/inspect sagas

Other best practices:- Make compensating actions explicit, small, and idempotent- Keep side effects localized and publish events via outbox- Version saga definitions to allow rolling upgrades- Write tests for normal, transient-failure, and multi-step-failure scenarios; include chaos tests for partial outages

This approach balances reliability, clear failure handling, and strong observability while keeping services decoupled and maintainable.

MediumTechnical

44 practiced

Compare orchestration vs choreography when implementing distributed workflows (sagas). For a high-throughput system with many short transactions, which approach would you prefer and why? Discuss maintainability, debuggability, and failure handling trade-offs.

Sample Answer

Short answer: for a high‑throughput system composed of many short transactions I’d generally prefer a choreography-first approach for performance and scalability, but with strict conventions and observability to mitigate its maintainability and failure‑handling downsides. For complex or business‑critical flows, use orchestration or a hybrid.

Comparison (concise):

- Architecture & throughput - Orchestration: single coordinator (service or workflow engine) drives each step. Easier to centralize logic and optimize retries, but creates a throughput/latency bottleneck and single point to scale and manage. - Choreography: each service reacts to events (pub/sub). Decentralized, highly parallel, lower latency and better horizontal scalability — a good fit for many short transactions.

- Maintainability - Orchestration: flow logic lives in one place → easier to change and reason about, simpler versioning. - Choreography: logic scattered across services → harder to understand whole flow; requires rigorous contracts and documentation.

- Debuggability & observability - Orchestration: easier to trace the end‑to‑end state in the orchestrator; fewer hops for root cause analysis. - Choreography: requires distributed tracing, correlation IDs, event catalogs and tooling to reconstruct flows; otherwise debugging is painful.

- Failure handling - Orchestration: straightforward to implement compensating transactions, backoff, and global timeouts since coordinator controls state. - Choreography: must design idempotent handlers, out‑of‑order protection, and compensating events; failure modes are more complex and eventual consistency is the norm.

Recommendation and practices:- Prefer choreography when latency and throughput are primary and transactions are short/simple.- Use a hybrid: orchestrate business‑critical multi‑step sagas and choreograph simple, independent event flows.- Compensations & reliability: make all handlers idempotent, publish explicit "failed" and "compensate" events, use DLQs.- Observability: mandatory distributed tracing with correlation IDs, structured logs, and an events catalog/dashboard to reconstruct flows.- Operational controls: enforce schema/versioning in events, circuit breakers, and rate‑limits; monitor end‑to‑end SLAs and compensating rates.

Trade‑off summary: choreography wins on throughput and resilience to load but costs more in maintainability and failure complexity; orchestration simplifies reasoning and error handling but risks scalability and single‑point complexity. Choose based on transaction complexity, throughput needs, and how much investment you can make in observability and tooling.

EasyTechnical

48 practiced

Compare and contrast a monolithic architecture with a microservices architecture. Discuss real-world trade-offs when choosing one over the other for a product expected to grow from 100 to 100k daily active users over two years. Cover operational complexity, deployment velocity, fault isolation, testing, and team organization.

Sample Answer

Definition / short summary:- Monolith: single deployable codebase (one process or tightly coupled modules). Simple to develop and deploy initially.- Microservices: many small, independently deployable services communicating over network (HTTP/gRPC, messaging). More modular but operationally heavier.

Compare by dimension:- Operational complexity: Monolith low — one deployment pipeline, simpler monitoring. Microservices high — service discovery, API gateways, distributed tracing, more infra (k8s, service mesh).- Deployment velocity: Monolith can be faster early (less coordination). Microservices enable independent deploys per service, increasing parallel velocity as team count grows.- Fault isolation: Monolith risk of cascading failures inside process; microservices isolate failures if designed with circuit breakers and bulkheads, but network failures add complexity.- Testing: Monolith favors end-to-end and integration tests within one process; microservices require contract testing, integration tests across services, and robust staging to catch distributed issues.- Team organization: Monolith suits small teams (1–5) where shared codebase eases collaboration. Microservices align with feature teams owning services (Conway’s law) and scale better for many teams.

Real-world trade-offs for growth 100 → 100k DAU in 2 years:- Start with a well-structured modular monolith (clear module boundaries, automated tests, CI/CD). Low ops cost and fastest time-to-market for early traction.- As traffic and team size increase, extract bottlenecked modules into services guided by performance metrics and team boundaries. Prioritize services that need independent scaling (payment, search, realtime).- Avoid premature microservices: unnecessary complexity can slow features and increase incidents.- Plan for observability, automated provisioning, and contracts early so extraction is smoother.

Recommendation:- Phase: Monolith → modular monolith with strong tests and CI → incremental decomposition into microservices when clear scaling or organizational needs justify the ops cost.

HardSystem Design

56 practiced

Design a globally distributed, multi-region service where users in any region can read and write their own data with low latency. Discuss replication topology, consistency guarantees, conflict resolution, leader selection, and how you would balance latency vs strong consistency for reads and writes across regions.

Sample Answer

Requirements & constraints:- Low-latency reads/writes from any region for that region’s users.- Global durability and cross-region availability.- Support mixed consistency: low-latency eventual reads, optional strong reads/writes.- Operate at shard/key granularity; assume large user base and many partitions.

High-level approach:- Partition data (user-id keyed) into shards. Each shard is replicated to N replicas across multiple regions.- Use a hybrid replication topology: region-local quorum (fast) + cross-region async replication with configurable consistency policies.

Replication topology:- For each shard, maintain R replicas, with at least one replica in each active region where users exist. Organize replicas into regional groups (local replica set) and a global membership.- Two common modes: 1. Single-Primary per-shard (region-aware leader): Leader normally placed in the user's home region (or the region with most owners). Synchronously replicate to local-region followers (faster), asynchronously replicate to remote regions. 2. Multi-leader/conflict-free: Allow leaders in multiple regions using CRDTs or operation logs with causal+version vectors for commutativity where possible.

Consistency guarantees & modes:- Default: Session-causal + read-your-writes within region using local leader/followers (writes ack to local quorum quickly). Cross-region replication asynchronous → eventual global convergence.- Strong (linearizable) option: Client request can target the shard’s global leader (or do a cross-region quorum write/read). Implemented via global quorum (W + R > N) or by proxying to the primary leader and waiting for acknowledgement from majority across regions. Offer this as per-operation policy (e.g., payments require strong).

Conflict detection & resolution:- If single-primary: conflicts rarely occur. For failover during leader transfer, use change vectors/operation IDs to reconcile.- If multi-leader: use CRDTs for commutative data types (counters, sets) where possible. For complex objects: - Attach vector clocks or logical timestamps (Lamport) to updates. - Default automated resolution: last-writer-wins with hybrid logical clock (HLC) to reduce clock skew issues. - Prefer application-level merge hooks for rich semantics: store divergent versions and invoke deterministic merge or require user/repair workflow.- Anti-entropy/gossip to reconcile divergence, with tombstones and GC windows to avoid resurrection.

Leader selection & failure:- Per-shard consensus (Raft/Paxos) within replica set for primary election. To reduce cross-region latency for election, prefer to run majority-capable nodes across regions but pin leader to a region via lease mechanism (e.g., leader prefers local region until it fails).- Use region-local fast path: client writes to local replica which proxies to current leader. If local leader not available, fall back to local follower that can serve reads or initiate leader election.- Global metadata service (small Paxos/etcd cluster) holds shard-to-leader mapping and cluster topology; use caching to avoid latency.

Balancing latency vs strong consistency:- Expose per-request consistency levels: - "Local-fast" (R_local quorum): low-latency reads/writes, eventual cross-region consistency. - "Global-strong" (global majority or leader-ack): higher latency, linearizable.- Use session affinity and sticky routing to home region to provide read-your-writes without global coordination.- Use leases/HLCs to provide bounded staleness reads (e.g., monotonic reads with staleness <= X ms) — read from replicas with last-applied timestamp >= now - X.

Operational considerations:- Monitor replication lag and provide backpressure if lag exceeds thresholds.- Anti-entropy and compaction to reclaim tombstones; maintain per-shard vector clocks and version histories for a limited window for reconciliation.- Security: encrypt replication channels, authenticate regions.- Testing: chaos/partition testing, simulate region failure, verify conflict resolution semantics.

Trade-offs summary:- Single-primary simplifies consistency and reduces conflicts but increases cross-region write latency for strong writes if leader is remote.- Multi-leader + CRDTs gives low-latency local writes but pushes complexity to conflict resolution and app-level merging.- Offering tunable consistency per operation lets you serve low-latency user interactions while preserving strong guarantees for critical paths.

HardSystem Design

51 practiced

Design an architecture for a payment settlement flow that must be atomic across internal services (ledger, anti-fraud, notification) and an external bank API. Explain trade-offs between using two-phase commit (2PC), sagas with compensations, or an external transaction coordinator. Address idempotency, retries, and eventual consistency guarantees.

Sample Answer

Requirements:- Strong atomicity: either ledger debit/credit, anti-fraud decision, notification and external bank transfer all succeed, or the user-visible result is rolled back or compensated.- Low latency for user-facing success; high reliability; external bank API is flaky/uncontrollable.- High throughput and auditability.

High-level architecture:- Orchestrator service (or external coordinator) that drives the payment workflow.- Per-service local transaction boundaries: Ledger service, Anti-fraud service, Notification service, and an External Bank Adapter.- Durable event store / saga log that records each step and its idempotent command IDs.

Approaches, trade-offs:1) Two-Phase Commit (2PC)- How: Use XA-like distributed transaction across services and bank (prepare/commit).- Pros: Synchronous atomicity, simple mental model.- Cons: Blocks resources (locks), poor availability if coordinator or bank hangs, most external banks don’t support prepare/commit, limited scalability. Not practical in microservices + external APIs.

2) Sagas with compensations (recommended)- How: Orchestrated saga pattern: orchestrator issues steps: reserve ledger funds (ledger: hold), call anti-fraud (approve/decline), call bank transfer (execute), finalize ledger (capture), notify customer.- If a later step fails (e.g., bank transfer), run compensating actions: release ledger hold, record reversal entries, send failure notification. Persist saga state in event store for retry.- Pros: Highly available, scalable, tolerant of partial failures, works with external HTTP APIs.- Cons: Only eventual consistency; requires careful compensation logic and idempotent ops; business complexity (e.g., partial money movement at external bank may be irreversible — must detect and reconcile).

3) External Transaction Coordinator / Hybrid- How: External durable coordinator (workflow engine — e.g., Temporal, Cadence) orchestrates saga steps and provides retry, timeouts, and long-running workflows.- Pros: Simplifies orchestration, reliable retries, visibility, built-in idempotency primitives and dead-letter handling.- Cons: Adds dependency and operational complexity.

Idempotency & retries:- All external-facing endpoints accept an idempotency key (UUID tied to payment attempt). Operations must be idempotent: ledger holds/captures use idempotency keys; bank adapter de-duplicates requests and stores external reference IDs.- Retry policies: exponential backoff with jitter for transient failures; circuit breaker for downstream instability; configured max attempts then escalate to manual reconciliation.

Eventual consistency & reconciliations:- Maintain audit log and periodic reconciliation job between ledger and bank statements. For irreversible bank transfers that partially complete, use reconciliation to create compensation transactions and mark sagas as “requires manual review” if automated compensation cannot guarantee correctness.- Use compensating transactions that are ledger-native (reversals) and customer-facing notifications marking state clearly.

Example flow (orchestrated saga):1. Create Payment (idempotency_key) -> ledger.create_hold(amount, idempotency_key)2. antiFraud.check(hold_id)3. bankAdapter.transfer(hold_id, idempotency_key) — store external_tx_id4. ledger.capture(hold_id, external_tx_id)5. notify.success(user)

On bank failure:- bankAdapter fails -> orchestrator -> ledger.release_hold(hold_id) (compensate) -> notify.failure(user) -> persist failure & schedule reconciliation.

Key operational practices:- Strong observability: traces, durable saga state, alerting for stuck sagas.- Short holds and TTLs to limit exposure.- Clear SLAs for manual reconciliation and dispute handling.

Recommendation:Use an orchestrated saga implemented with a durable workflow engine (Temporal/Cadence) + strict idempotency keys, compensating transactions, and robust reconciliation for edge cases where external bank behavior is non-atomic. This balances availability, scalability, and practicality for real-world bank integrations.

Unlock Full Question Bank

Get access to hundreds of Design Pattern Application and Trade-offs interview questions and detailed answers.

Join thousands of developers preparing for their dream job.