Decision Making Under Uncertainty Questions

Focuses on the frameworks, heuristics, and judgment used to make timely, defensible choices when information is incomplete, conflicting, or still evolving, in any domain. Covers diagnosing what is genuinely unknown before deciding, setting explicit decision criteria and thresholds, weighing probabilities against impact (expected value and cost benefit thinking), and defining upfront triggers for reversing course, escalating, or waiting for more evidence. Also covers calibrating risk tolerance to the stakes involved, choosing between a small test or pilot versus committing directly to a decision, communicating uncertainty and trade offs to stakeholders in plain terms, and how senior candidates fold organizational constraints (budget, time, politics, precedent) into a call when the fully right answer cannot be known in advance. The underlying judgment applies to any high-stakes decision made with partial information: a hiring call with an incomplete reference check, a budget reallocation with uncertain ROI, a legal or compliance risk judgment, a vendor or partner selection, a go/no-go on a product bet, or a technical rollout. No single domain should dominate the framing.

HardTechnical

53 practiced

You are considering enabling backup compression to reduce storage costs, but restores will take 3x longer and consume additional CPU during restore. Using RTO/RPO requirements and an expected restore frequency, perform a quantitative cost-benefit analysis and state the criteria under which you would enable compression for production backups. Include considerations for mixed strategies (e.g., compressed cold archives vs uncompressed recent backups).

Sample Answer

**Approach summary**I quantify storage savings vs restore cost (time and CPU) using RTO/RPO and expected restore frequency (annual). I show decision thresholds and a mixed strategy for production.

**Key variables**- S = current backup storage size (TB)- c_s = storage cost per TB per month ($)- f = expected restores per year- t_un = average restore time uncompressed (hours)- t_comp = restore time compressed = 3 * t_un- c_cpu = incremental CPU cost per restore ($) — include cloud VM cost or opportunistic ops labor- y = years considered- p = compression ratio (e.g., 0.4 = 60% savings)

text

Annual storage savings = S * (1 - p) * c_s * 12 * y
Annual restore cost delta = f * ( (t_comp - t_un) * hourly_op_cost + incremental_CPU_cost_per_restore )
Net benefit = Annual storage savings - Annual restore cost delta

**Example (numbers)**- S = 10 TB, c_s = $25/TB/month, p = 0.4 => savings = 10*(1-0.4)*25*12 = $1,800/year- t_un = 2 hr, t_comp = 6 hr, hourly_op_cost = $50 (worker or provisioned VM), f = 4 restores/year- Restore delta = 4 * (4 hr * $50) = $800/year (+ any added CPU billed say $200) => $1,000- Net = $1,800 - $1,000 = $800 net positive -> enable compression

**Decision criteria**Enable compression if:- Net benefit > 0 over your planning horizon AND- Compressed restore time (t_comp) <= RTO for affected recovery tier OR you can mitigate with warm standby; AND- RPO unaffected; backups still verify successfully; AND- CPU/IO spikes during restore are tolerable (no SLA impact).

**Mixed strategy**- Keep last N days/weeks uncompressed for fast restores (meet strict RTO); compress older backups (cold archive) to save long-term cost.- Use tiered storage: compressed → cheaper object storage; recent uncompressed on fast block storage.- Automate lifecycle: after X days, recompress and move to cold tier; document runbooks for longer restore times.

**Operational considerations**- Test restore times and CPU usage; include failover rehearsals in SLA calculations.- Monitor actual restore frequency and revisit thresholds annually.

MediumSystem Design

51 practiced

You're on-call and receive alerts that Region-A is seeing increased 5xx errors and latency, while Region-B reports healthy metrics. There is risk that shifting traffic to Region-B could overload it. With a 15-minute RTO target, describe the decision criteria you would use to decide whether to shift traffic, and provide a step-by-step failover plan including canary traffic shifts, health checks, rollback triggers, and stakeholder communication.

Sample Answer

**Decision criteria (first 3 min)**

- Severity: sustained 5xx rate > 1% or latency p95 > SLO for > 2 consecutive 1-min windows in Region-A. - Impact: user-visible errors or degraded UX (orders failing, auth errors). - Capacity: Region-B remaining capacity >= 40% headroom (based on current utilization + estimated shifted load). - Dependency health: upstream services (DB, cache, third-party) globally healthy or limited to Region-A. - RTO alignment: can complete safe traffic shift, verification, and rollback within 15 minutes.

**Failover plan (step-by-step)**

1. Triage (0–2 min) - Acknowledge alert, open incident channel, notify on-call lead and SRE manager. - Confirm Region-B metrics and check autoscaling, network, and global services.

2. Prepare (2–4 min) - Lock change control to on-call. - Pre-stage DNS / LB configuration and runbook; ensure runbook accessible.

3. Canary shift (4–9 min) - Shift 5% traffic from Region-A → Region-B via weighted LB or traffic-splitting flag. - Monitor per-second health: error rate, latency p95, CPU, memory, queue depth. - Health checks: application 200 response, dependency latencies, and business transactions (e.g., login, payment sandbox).

4. Evaluate (9–11 min) - Success criteria for canary: no increase in 5xx or latency > SLO, CPU < 75%, no queue growth for 2 consecutive minutes. - If success, increment to 20% and repeat 1–2 min observation.

5. Full shift (11–13 min) - If progressive canaries stable, move to 50% then 100% or until Region-A isolation required. - Enable autoscaling in Region-B if not already.

6. Rollback triggers (anytime) - Any Canary/step shows: 5xx > baseline + 0.5%, latency p95 > SLO, CPU > 85%, or dependency errors rising — immediate revert to last safe weight. - If Region-B unstable after revert, route traffic to degraded-mode endpoints or enable rate-limiting and engage engineering.

7. Communication (continuous) - Post status updates every 5 minutes in incident channel: actions taken, metrics, decision. Notify Product and Support once user-impact confirmed and after mitigation. - After resolution, write incident report with timeline, root cause hypotheses, and remediation actions.

**Notes / safeguards**- Use short TTL DNS only as fallback; prefer LB weighted routing for fast rollbacks.- Automate metric dashboards and alarms for canary windows.- Keep RTO by limiting steps and predefined weights; prioritize safety over complete failover if uncertainty exists.

MediumTechnical

39 practiced

Case study: Your company wants to roll out a multi-region deployment for a latency-sensitive stateless API. Create a 5-step decision plan that includes pilot design (traffic slices), exact metrics to evaluate latency and correctness, rollback criteria, cost-estimation factors, and how to surface legal or compliance constraints per region.

EasyTechnical

42 practiced

Write a Bash script that accepts a process name and a space-separated list of hostnames as arguments, SSHes to each host (assume key-based auth), checks if the process is running using pgrep or ps, and prints 'hostname: running' or 'hostname: not-running' for each host. Keep the script under 40 lines and include a simple SSH timeout so a slow host doesn't block the whole check.

Sample Answer

**Approach**- Use a short Bash script that takes the process name as first arg and remaining args as hostnames.- Use the timeout utility to avoid long waits, and ssh options for fast fail on auth/network.- On remote host check with pgrep (preferred) falling back to ps if pgrep absent.

bash

#!/usr/bin/env bash
# checkproc.sh: check process on multiple hosts
# Usage: ./checkproc.sh process_name host1 host2 ...

set -euo pipefail

if [ $# -lt 2 ]; then
  echo "Usage: $0 process_name host1 [host2 ...]" >&2
  exit 2
fi

proc=$1; shift

for host in "$@"; do
  # overall timeout for each host (seconds)
  if timeout 8 ssh -o BatchMode=yes -o ConnectTimeout=4 "$host" \
       "pgrep -f -- '$proc' >/dev/null 2>&1 || ps aux | grep -v grep | grep -q -- '$proc'"; then
    echo "$host: running"
  else
    echo "$host: not-running"
  fi &
done
wait

Explanation / notes- Uses timeout 8s to cap total SSH+remote command time per host.- ConnectTimeout=4 limits TCP connect phase; BatchMode avoids password prompts.- Runs checks in parallel (background + wait) so many hosts don't serially block the check.- Adjust timeouts as needed for high-latency networks.

HardTechnical

42 practiced

You discover a latent bug that corrupts roughly 1 in 10,000 user records during the write path. The root cause is unknown. Options include rolling back the release (losing some recent features), shipping a quick mitigation, or continuing while investigating. Perform a quantitative decision analysis that considers probability of detection, expected number of affected users, business/regulatory impact, cost to remediate corrupted data, and rollback risk. Recommend an action and justify it.

Unlock Full Question Bank

Get access to hundreds of Decision Making Under Uncertainty interview questions and detailed answers.

Join thousands of developers preparing for their dream job.