Decision Making Under Uncertainty Questions

Focuses on the frameworks, heuristics, and judgment used to make timely, defensible choices when information is incomplete, conflicting, or still evolving, in any domain. Covers diagnosing what is genuinely unknown before deciding, setting explicit decision criteria and thresholds, weighing probabilities against impact (expected value and cost benefit thinking), and defining upfront triggers for reversing course, escalating, or waiting for more evidence. Also covers calibrating risk tolerance to the stakes involved, choosing between a small test or pilot versus committing directly to a decision, communicating uncertainty and trade offs to stakeholders in plain terms, and how senior candidates fold organizational constraints (budget, time, politics, precedent) into a call when the fully right answer cannot be known in advance. The underlying judgment applies to any high-stakes decision made with partial information: a hiring call with an incomplete reference check, a budget reallocation with uncertain ROI, a legal or compliance risk judgment, a vendor or partner selection, a go/no-go on a product bet, or a technical rollout. No single domain should dominate the framing.

MediumTechnical

52 practiced

Estimate the expected value of reducing the payment microservice error rate from 1% to 0.1%. List all inputs you need (average revenue per transaction, conversion impact per failed payment, engineering cost, rollout risk), how you would gather or estimate them, and how you would handle large uncertainty in those inputs.

Sample Answer

Approach summary: treat this as an expected-value (EV) cost‑benefit problem: estimate incremental revenue recovered from fewer failed payments, subtract engineering cost and rollout risk-adjusted losses. Present inputs, how to get them, an example calculation, and ways to handle uncertainty.

Required inputs- Current transaction volume (Tx/day or Tx/year). Source: analytics (Segment, GA, data warehouse).- Average revenue per successful transaction (ARPT). Source: finance / order data = total GMV / #successful Tx.- Baseline failure rate (1%) and target failure rate (0.1%). Source: payments logs / SRE metrics.- Conversion impact per failed payment (what fraction of failed payments are retried/completed vs lost). Source: event funnel analysis: track failed-payment → retry → success within X hours; or cohort analysis of user behavior after failure.- Engineering cost to implement fix (one‑time dev + QA + infra + monitoring). Source: engineering estimates (T-shirt sizing → story points → cost using loaded hourly rate).- Ongoing maintenance cost (annual). Source: engineering/ops estimates.- Rollout risk / negative side-effects probability and impact (e.g., regressions causing downtime or fraud). Source: past incident rates, design review, risk assessment.- Time horizon (months/years) and discount rate.

How to gather/estimate them- Query data warehouse for Tx counts, revenues, and failure events.- Instrument event pipeline to tag failure and retry paths; run 30–90 day cohort analysis.- Run A/B or gradual rollout to measure conversion lift (if feasible).- Get engineering estimates via RFC and planning poker; include contingency (20–30%).- Estimate risk by consulting SRE/Payments SMEs and historical incident severity.

Calculation templateEV = (DeltaSuccessfulTx) * ARPT * (1 − marginal churn after failure) * time_horizon − EngineeringCost − ExpectedRiskCost − MaintenanceCost

Where DeltaSuccessfulTx = TotalTx * (baseline_rate − target_rate)

Numeric example (annual, simple):- TotalTx/year = 10,000,000- ARPT = $50- Baseline−target = 0.01 − 0.001 = 0.009- If a failed payment loses 80% of revenue (only 20% retry), effective gain fraction = 0.8Gross benefit = 10,000,000 * 0.009 * $50 * 0.8 = $3,600,000Subtract: Engineering one‑time $400k + annual maintenance $50k + expected rollout risk cost (probability 5% * impact $500k = $25k) → Net ≈ $3.125M first year.

Handling large uncertainty- Run sensitivity analysis: vary ARPT, Tx, retry rate, and engineering cost across plausible ranges and show best/worst/case.- Monte Carlo simulation to produce confidence intervals for EV.- Prioritize quick experiments (small A/B or canary) to measure retry behavior and conversion lift before large build.- Use decision rules: if conservative (pessimistic) EV > cost by X margin (risk appetite), approve; else stage investment (pilot → iterate).- Consider low-cost mitigations first (improve retry logic, better error messages, observability) to reduce uncertainty quickly.

Recommendation: gather the key analytics (Tx, ARPT, real retry rate) and run a 2‑week pilot/canary. If conservative EV > 2× engineering cost, proceed with full implementation.

HardSystem Design

46 practiced

You must decide whether to shard a social network by user-id or by geographic region. Traffic distribution is partially unknown and likely heavy-tailed. Describe the decision process, how you would gather additional data quickly, design a migration and rollback plan, and handle hot shards or uneven growth under uncertainty.

Sample Answer

Requirements & constraints:- Functional: low-latency reads/writes, consistent user experience globally, support follow/feeds, location-based queries.- Non‑functional: handle heavy‑tailed traffic, predictable scaling, minimal user-visible migration downtime, rollback capability.- Unknowns to reduce: skew by user-id vs region, multi-region read/write patterns, growth trends.

Decision process:1. Define evaluation metrics: per-shard QPS, P95 latency, cache hit-rate, cross-shard fan-out cost, operational cost.2. Hypothesis: user-id sharding balances users uniformly if signups are uniform; region sharding reduces cross-region latency and legal data locality but risks hot regions.3. Run experiments and gather data before committing.

Quick data gathering:- Instrument existing production for short window (24–72h) to capture: - Per-user and per-region request rates, active users, session patterns. - Cross-region traffic matrix (where requests originate vs data owner). - Fan-out sizes (followers distributions). - Growth cohort projections.- Use sampling & aggregation to avoid overhead; backfill logs for recent weeks.- Run A/B experiments: route a small percentage of traffic to prototype sharding schemes in staging/Canary to measure cross-shard cost and latency.

Design migration plan:- Phased approach: 1. Preparation: build mapping service (user-id→shard or region→shard), dual-write abstraction in API layer, idempotent writes, and a background rewriter. 2. Dry run: migrate read-only historical snapshots to target layout in staging; verify queries and fan-out behavior. 3. Canary: pick low-risk user cohort (e.g., new users or low-activity region). Enable routing to new shards, dual-read for verification. 4. Gradual ramp: expand cohorts by traffic percentiles, monitor SLOs and fan-out costs. 5. Cutover: finalize mapping, enable single source writes, decommission old shards.

Rollback plan:- Always maintain dual-write and dual-read during migration window so rollback is switching mapping and continuing dual operations until safe.- Keep backwards-compatible schemas and replay logs to rehydrate old layout if needed.- Define abort criteria (e.g., >10% latency regression, error spike, queue backlog) and automated traffic re-routing to previous layout.

Handling hot shards / uneven growth:- Design shards as logical groups with ability to split: consistent hashing with virtual nodes, or tenant-aware shard key that supports split/merge.- Use a routing layer (shard map) that supports moving virtual nodes without global downtime; background data copy with write-forwarding.- Employ cache & CDN for reads; write-back queues and rate limiting for heavy fan-outs; prioritize critical operations.- Autoscaling: add capacity to hot shard’s hardware and isolate noisy neighbors via resource quotas.- Long-term: monitor signals, adopt hybrid scheme (primary shard by region for latency-sensitive reads, secondary hash by user-id for write distribution), or use a two-level sharding (region → user-id buckets) to balance locality and evenness.

Communication & product considerations:- Align stakeholders: engineering (migration ops), legal (data locality), support (rollback playbook), marketing (communication to affected users).- Schedule migrations during low-traffic windows, prepare customer-facing messaging for any expected degraded features.- Post-migration: report metrics vs targets, iterate on sharding strategy if growth invalidates assumptions.

Trade-offs:- User-id sharding: more uniform distribution, worse cross-region latency.- Region sharding: better latency/data locality, risk of hotspots.- Hybrid/two-level sharding increases complexity but hedges uncertainty.

This plan minimizes user impact, provides reversible steps, collects actionable data quickly, and builds operational tools (routing, split/merge) to handle heavy‑tailed growth.

EasyTechnical

39 practiced

Given incomplete telemetry about adoption and activation of a new feature, explain how you would use expected-value reasoning to decide whether to delay launch or proceed. Provide a short worked example: estimate probabilities, impact on conversion, engineering cost, and show the expected-value calculation you would present to stakeholders.

Sample Answer

Approach (framework): Frame the choice as an expected-value (EV) comparison between two actions: Launch now vs Delay (to collect more telemetry). Estimate probabilities for scenarios (feature works as expected / underperforms / causes harm), quantify conversion impact and costs for each, compute EV = sum(prob_i * outcome_i) minus costs, and compare. Include value of information (VOI): extra data may reduce uncertainty; delay is worth it if EV gain > delay cost.

Worked example:Assumptions (30-day horizon; monthly active users = 100,000; baseline conversion = 5% -> 5,000 conversions; revenue per conversion = $50):Scenarios if we Launch now:- Success (improves conversion by +1.5ppt to 6.5%): P=0.5 → conversions = 6,500 → incremental revenue = (1,500 * $50) = $75,000- Neutral (no change): P=0.3 → incremental revenue = $0- Harm (reduces conversion by -1.0ppt to 4.0%): P=0.2 → loss = (1,000 * $50) = $50,000Engineering launch cost (one-time) = $20,000. Total EV(Launch) = 0.5*$75,000 + 0.3*$0 + 0.2*(-$50,000) - $20,000 = $37,500 + $0 - $10,000 - $20,000 = $7,500

If we Delay (2 weeks) to run a small experiment / fix telemetry:- Cost of delay: missed half-month revenue opportunity if feature would have succeeded = expected gain lost ≈ 0.5*(75,000/2) = $18,750 (approx)- Additional eng & analytics cost to validate = $10,000- But uncertainty reduces: success prob increases to 0.7, harm to 0.1, neutral 0.2.Expected benefit after delay and then launch: 0.7*$75,000 + 0.2*$0 + 0.1*(-$50,000) - $20,000 (launch cost) - $10,000 (validation) - $18,750 (delay opportunity cost) = $52,500 + $0 - $5,000 - $20,000 - $10,000 - $18,750 = -$1,250

Decision and reasoning:- EV(Launch now) = +$7,500; EV(Delay & validate) ≈ -$1,250. Based on these estimates, proceed to launch now.- Sensitivity: if harm probability higher or loss bigger, or validation cheaper / faster, delay could win. Present these assumptions and a sensitivity table to stakeholders (vary P(success) and harm magnitude).Recommendation: launch now but instrument quickly, run a rapid rollback plan and close monitoring for first 48–72 hours. If telemetry shows harm, revert fast. If stakeholders prefer caution, propose a short A/B roll-out (e.g., 10% traffic) to get data with lower downside — show its EV similarly.

HardTechnical

43 practiced

You're planning a multi-month rewrite of a core service. Propose a phased plan including decomposition into bounded contexts, decision milestones, integration test strategy, feature parity acceptance criteria, risk windows, and how you'd make go/no-go decisions at each milestone under uncertainty.

Sample Answer

Approach: break the rewrite into phased, measurable increments that reduce blast radius, preserve customer value, and surface technical/market uncertainty early. Use bounded contexts to decompose the service, time-boxed milestones with clear decision gates, and automated integration tests plus targeted risk windows.

Phase 0 — Discovery (4–6 weeks)- Map current domains, dependencies, and data contracts; identify bounded contexts (e.g., Auth, Billing, Catalog, Search).- Define non‑functional targets (latency, error budget), feature-parity matrix, and KPIs.Decision gate: go/no-go to implementation based on readiness of contracts, QA resources, and migration plan.

Phase 1 — Strangler Facade & Core APIs (8–12 weeks)- Implement new core APIs behind a facade; route a small % of traffic.- Bounded context: start with the lowest-risk domain (read-only or batch workflows).Integration test strategy: comprehensive contract tests (consumer-driven contracts), end-to-end smoke tests in staging, canary pipelines.Acceptance: 100% behavior for selected surfaces; <=1% increase in latency; automated contract tests pass.Risk window: 2 weeks of canary at production traffic segment.Decision gate: if SLA breaches, contract test failures, or >threshold errors, pause and roll back.

Phase 2 — Incremental Migration (multiple 6–8 week sprints)- Migrate additional contexts iteratively (write-heavy next). For each: design adapter, migrate schema with dual-write or backfill pattern.- Tests: integration, data-sync verification, backward-compatibility tests, chaos testing in pre-prod.Acceptance: feature parity for migrated endpoints, data consistency within defined RPO/RTO, observability dashboards green for 2 full business cycles.Risk window: longer monitoring (4 weeks) for write domains.Decision gate: use a Decision Matrix combining quantitative signals (error rate, latency, data drift, user KPIs) and qualitative readiness (ops playbooks, rollback tested). If metrics exceed thresholds, halt progression.

Phase 3 — Cutover & Decommission (4–8 weeks)- Flip traffic, deprecate old code, run post-mortem and performance tuning.- Final acceptance: all contracts on new platform, no critical bugs for 30 days, stakeholder sign-off.Decision: only proceed when error budget consumption acceptable and support/incident runbooks validated.

Go/No-Go framework under uncertainty- Use predefined quantitative thresholds (error rate >X, latency >Y, data delta >Z) and a RAG scoring for readiness (Testing, Monitoring, Rollback, Team Capacity).- Apply staged rollback plans and short risk windows to limit exposure.- Escalation: if borderline, apply a “pause-and-harden”—delay next migration, allocate 1–2 sprints for fixes and stronger telemetry.- Communication: weekly stakeholder dashboards with top-line KPIs and risk register; executive sign-off required for high-impact domains.

Key trade-offs: speed vs. safety — mitigate via canaries, consumer-driven contracts, strong observability, and business-prioritized order of bounded contexts.

EasyTechnical

50 practiced

Explain how you would build and use a probability-impact heatmap to prioritize architectural risks for a high-traffic service. Provide at least three example risks (e.g., data loss, latency spike, vendor outage) and show how the heatmap guides decisions under limited data.

Sample Answer

Start by clarifying objectives and constraints: we're prioritizing architectural risks for a high-traffic service where engineering capacity is limited and data about failure likelihood is sparse.

How to build the probability-impact heatmap1. Define axes and scales: Probability (Low/Medium/High or 1–5) and Impact (Minor/Moderate/Critical or 1–5). Use qualitative definitions tied to business metrics (e.g., impact = revenue loss per hour, SLA breaches, or user churn).2. List candidate risks and estimate scores using available evidence (logs, past incidents, vendor SLAs, expert judgement). Capture uncertainty ranges.3. Plot each risk on the 5x5 grid and color zones (green/yellow/red). Add annotations for confidence level.4. Produce an action matrix: Accept/Monitor/Mitigate/Immediate Action mapped to heatmap zones.

Example risks and scoring (illustrative)- Data loss: Probability = 2 (rare backups tested yearly), Impact = 5 (irreversible user data loss → legal/reputation). Plotted in high-impact, low-probability (red edge) → prioritize mitigation (improve backup verifiability, DR runbooks).- Latency spike: Probability = 4 (peak traffic variability), Impact = 3 (degrades UX, conversions). Medium-high zone → prioritize detection & quick mitigations (circuit breakers, autoscaling, synthetic tests).- Vendor outage (CDN/auth provider): Probability = 3, Impact = 4 (partial degraded functionality). High-medium zone → mitigation: multi-vendor fallback, graceful degradation.

Using the heatmap under limited data- Use expert opinion + conservative scoring for impact; express probability as range and adopt the upper bound for prioritization when uncertainty is high.- Run low-effort experiments to reduce uncertainty (chaos engineering for latency, backup restore drills for data loss). Re-score after results.- Allocate effort proportional to expected business exposure: focus first on risks with high impact × probability and low confidence because they offer the biggest value from small investments in validation.

Outcome and communication- Share the heatmap with stakeholders, including confidence bands and recommended next steps (mitigate, monitor, accept). Use it as a living artifact—re-score after tests or incidents to keep the roadmap aligned with risk reality.

Unlock Full Question Bank

Get access to hundreds of Decision Making Under Uncertainty interview questions and detailed answers.

Join thousands of developers preparing for their dream job.