Architecture and Technical Trade Offs Questions

Centers on system and solution design decisions and the trade offs inherent in architecture choices. Candidates should be able to identify alternatives, clarify constraints such as scale cost and team capability, and articulate trade offs like consistency versus availability, latency versus throughput, simplicity versus extensibility, monolith versus microservices, synchronous versus asynchronous patterns, database selection, caching strategies, and operational complexity. This topic covers methods for quantifying or qualitatively evaluating impacts, prototyping and measuring performance, planning incremental migrations, documenting decisions, and proposing mitigation and monitoring plans to manage risk and maintainability.

HardTechnical

29 practiced

Compare synchronous two-phase commit (2PC) and eventual-consistency (sagas/compensating transactions) approaches for cross-service transactions such as order placement affecting inventory and billing. Discuss developer complexity, latency and availability impacts, common failure modes, and how you'd decide which best fits product requirements.

Sample Answer

High-level summary- 2PC: distributed locking protocol that gives strong consistency (all participants commit or all abort) but requires synchronous coordination and a coordinator.- Eventual-consistency/sagas: each service performs local transactions and, on failure, runs compensating actions to undo prior steps; consistency converges over time.

Developer complexity- 2PC: relatively low application logic change — transactional semantics are centralized — but requires infrastructure to run coordinator, participant proxies, and careful handling of blocking and resource locking. Hard to test at scale and to debug distributed deadlocks.- Sagas: higher application-level complexity: teams must design forward and compensating flows, idempotency, and reconciliation processes. But implementation fits microservice ownership: each team controls its local steps and compensations.

Latency and availability- 2PC: higher latency (multiple synchronous round-trips) and reduces availability — if coordinator or participant is slow/down, global transaction blocks (blocking protocol). Not suitable across high-latency networks or unreliable services.- Sagas: lower end-to-end synchronous latency (each step commits locally), much higher availability — operations proceed even if downstream services are temporarily unavailable; reconciliation happens asynchronously.

Common failure modes- 2PC: coordinator crash during commit/prepare, participant crash leaving locked resources, network partitions causing indefinite blocking; complex failure recovery.- Sagas: partial completion where some steps succeed and compensations fail or are delayed; causal ordering bugs; double-charges or temporary inventory inconsistencies; compensating actions may be imperfect or irreversible (business limitations).

How to decide (product-prioritized criteria)- Consistency requirement: If business cannot tolerate any inconsistency (e.g., banking settlement, legal obligations), prefer strong consistency (2PC or equivalent). For retail order placement where small windows of inconsistency are acceptable and can be reconciled, prefer sagas.- User experience & latency: If checkout latency must be low, sagas often better. For workflows where user waits for definitive final state, consider synchronous approach.- Availability SLAs: For high availability and resilience to partial outages, sagas win.- Operational cost & team ownership: If org is microservice-first with independent teams, sagas align better; 2PC introduces centralized ops burden.- Compensability: If all actions have reliable inverse operations, sagas are workable. If compensations are impossible or risky, lean toward strong coordination.- Metrics to validate decision: error rate of compensations, mean time to reconcile, user-facing inconsistency window, transaction latency, and revenue impact of inconsistent states.

Recommendation (example)For an e-commerce order that touches inventory and billing: use a saga pattern for normal UX (fast checkout, resilient), implement strong pre-authorization for billing and inventory soft-reserve to reduce compensations, add an automated reconciliation job and alerting for failed compensations. Reserve synchronous commit only for high-value orders or regulatory flows where atomicity is mandatory.

EasyTechnical

28 practiced

Explain eventual consistency in simple terms. Provide one product scenario where eventual consistency is acceptable and one where it is unacceptable. Describe specific user-visible issues that might arise and how you would mitigate or communicate those issues to users.

HardTechnical

37 practiced

Design an architecture roadmap that balances simplicity and extensibility for a platform expected to grow unpredictably. As PM, propose a 0-6 month, 6-18 month, and 18+ month plan, define decision criteria for when to refactor, and list architectural fitness functions you would use to avoid premature optimization while keeping options open.

Sample Answer

0–6 months (establish safe default)- Goals: deliver core product, validate key value props, minimize engineering debt, enable fast learning.- Tactics: single well-documented modular monolith or modular service scaffold (bounded modules, clear API contracts), feature flags, CI/CD, automated tests, telemetry (usage, latency, error), developer onboarding docs.- Outcomes: ship 2–3 core flows, <48h deploy turnaround, baseline KPIs for engagement/retention.

6–18 months (grow with controlled modularity)- Goals: enable parallel development, reduce coupling where it blocks velocity, instrument for scaling decisions.- Tactics: identify hot/volatile domains from telemetry and decompose into independently deployable services (strangler pattern). Introduce async messaging for cross-cutting concerns, stronger API versioning, SLOs, capacity planning. Invest in testing (contract tests), observability, and standardized infra templates.- Outcomes: independent deploys for 1–3 services, improved cycle time, measurable SLOs.

18+ months (scale and optimize for scale/complexity)- Goals: support unpredictable growth, cost-efficiency, and extensibility.- Tactics: selective polyglot or data partitioning where justified, platform engineering (self-service infra), CI/CD at scale, mature governance (API catalog, SLA contracts), cost observability, and refactor roadmap based on measured signals.- Outcomes: predictable scaling, clear ownership, ability to add major features with bounded impact.

Decision criteria to refactor (must meet multiple):- Reproducible velocity impact: >20% increase in cycle time or frequent merge conflicts in a module.- Operational pain: sustained outages or SLO breaches tied to architecture.- Cost inefficiency: >30% higher infra cost per MAU vs baseline for a component.- Business opportunity blockage: inability to deliver a prioritized revenue-generating feature within acceptable time/effort.- Measurable technical debt index exceeds threshold (test coverage, coupling score).Refactor only when benefits (reduced cycle time, reliability, cost, business velocity) outweigh immediate product opportunity cost — estimated ROI and rollback plan required.

Architectural fitness functions (keep options open, avoid premature optimization)- Incremental deployability: “Can team X ship feature Y independently?” (measure deploy frequency, lead time)- Coupling metric: code/module coupling score must trend downward after modularization decisions.- Latency & throughput SLOs: end-to-end 95th percentile latency bounds for core flows.- Error budget: rate of incidents tied to changes remains below threshold.- Cost per MAU trend control: surface cost regressions early.- Test/coverage guardrails: critical paths require min contract/integration test coverage.- Observability coverage: percent of services with full tracing, metrics, and dashboards.- Developer productivity: pull-request cycle time and mean time to onboard new dev.- API stability index: breaking changes rate below agreed threshold.

Why this balances simplicity & extensibility- Start with simple, fast-to-ship design to validate product-market fit; instrument heavily.- Use telemetry-driven, incremental decomposition to only pay complexity costs where justified.- Decision criteria and fitness functions ensure refactors are data-driven and aligned to business impact, preventing premature architecture work while keeping future options open.

HardSystem Design

30 practiced

Design a rollout strategy using feature flags and canary releases for a multi-service change that impacts payments and notifications. As PM, list gating criteria before increasing exposure, required observability, automated rollback triggers, and stakeholder communication steps to ensure safe deployment.

Sample Answer

Requirements / constraints:- Change touches payments (financial correctness, latency, reliability, compliance) and notifications (delivery, correctness, privacy).- Minimal customer impact, ability to quickly rollback, auditability for compliance, cross-service coordination across Payments, Notifications, API/Gateway, and monitoring.

Rollout strategy (high level):1. Feature-flag everything: server-side flags per service with versioned config and targeting rules.2. Canary release: start with 1% of traffic -> 5% -> 25% -> 100%, gated by metrics.3. Phased audience: internal QA -> SRE + biz QA -> small real customer cohort (non-high-value) -> all.

Gating criteria before increasing exposure (explicit, pass/fail thresholds):- Payments: - Successful payment rate ≥ baseline - 0.1% - Error rate (5xx/4xx related to change) ≤ baseline + 0.05% - Mean/95th latency ≤ baseline + 20% - No reconciliation discrepancies in sampled transactions; end-to-end settlement match- Notifications: - Delivery success rate ≥ baseline - 0.5% - Error/dropped messages rate ≤ baseline + 0.2% - Content correctness checks pass for sampled messages- Cross-service: - No increase in correlated errors across services - No GDPR/Pii leaks in logs - Automated integration test suite green- Business: - No increase in customer support tickets beyond defined threshold (e.g., +10% over baseline for canary window)

Required observability:- End-to-end transaction tracing (distributed traces propagated across payments & notifications)- Real-time dashboards: success/error rates, latency (p50/p95/p99), throughput, queue/backlog sizes, retry counts- Business KPIs: revenue per minute, declined payments, failed refunds, notification CTR where relevant- Alerting: anomaly detection (statistical and threshold-based) and SLO/SLA breach alerts- Sampling & log retention for full payloads for failed transactions (masked for PII)- Reconciliation job outputs and integrity checks surfaced

Automated rollback triggers (immediate rollback if any trigger trips):- Absolute thresholds: - Payment errors > X% (e.g., 0.5%) absolute increase vs baseline for rolling window (5m) - Payment latency p95 > baseline + 1000ms - Any failed settlement/integrity check for sampled transactions- Relative anomaly detection: - Statistically significant rise in payment declines or notification failures detected by anomaly engine- Business-safety triggers: - Revenue drop > Y% in window or spike in refunds/chargebacks- Dependency cascades: - Downstream queue/backlog exceeding safe limits for > 2m- On trigger: automated flag flip to rollback canary cohort, coupled with a circuit-breaker on gateway routing

Stakeholder communication and runbook:- Before rollout: notify engineering leads (Payments, Notifications), SRE, Legal/Compliance, Customer Support, Sales, and Product Marketing with timeline, rollback plan, and runbook link.- Canary start: “Canary live — monitoring active” message; SRE watches and stands by.- On each gate pass: short update to stakeholders and increment exposure.- On failure: automatic alert to on-call + PM initiates incident call (post-rollback) within 5 minutes; Customer Support provided with FAQ and mitigation script; Legal/Compliance looped if financial integrity affected.- Post-rollout: 24/72-hour post-mortem, metrics review, lessons & flagged changes.

Roles & responsibilities:- PM: owns gating criteria, stakeholder comms, business decision to proceed/stop.- Engineering: implement flags, observability, automated rollback.- SRE: monitoring, runbook execution, automation.- Support/Sales: customer comms and mitigation.

Trade-offs:- Slower ramp reduces risk but increases time-to-value.- More conservative thresholds reduce false positives but may delay rollout.

This plan ensures safe, auditable, and reversible deployment of multi-service changes impacting payments and notifications.

EasyTechnical

35 practiced

Explain the CAP theorem (consistency, availability, partition tolerance) in terms a non-engineer stakeholder can understand. Give two concrete product examples: one where you would prioritize consistency over availability and one where you would prioritize availability over consistency. For each example, describe the business impact and how you would communicate the trade-off to customers and internal stakeholders.

Unlock Full Question Bank

Get access to hundreds of Architecture and Technical Trade Offs interview questions and detailed answers.

Join thousands of developers preparing for their dream job.