Operational Risk Management Questions

Direct answer. On the numbers I would choose the $250k preventive control, because it removes the most loss and overtakes the cheaper option within two years. The cost figures are one-time for the controls and the $40k is per year, so the comparison depends on the horizon. I would also check the 90% claim, because the whole case rests on it.

Terms. A preventive control stops the bad event from happening (for example, blocking a data leak). A detection control notices it after it has started so you can respond (for example, an alert plus a response team). Expected annual loss is probability times impact per year. Residual loss is the expected loss still left after a control. Loss removed is the baseline minus the residual. Cumulative cost is everything spent so far plus the residual loss paid so far.

Assumptions. Baseline expected annual loss is $300k. The $250k and $100k are one-time costs; the $40k response cost recurs yearly. The preventive control has no running cost (a simplification; real controls have some). Figures in $k.

Calculations

Option A (preventive, 90% effective): residual loss = 300 x (1 - 0.90) = 30 per year
                       loss removed  = 300 - 30 = 270 per year
Option B (detect + contingency, 70% effective): residual loss = 300 x (1 - 0.70) = 90 per year
                       loss removed  = 300 - 90 = 210 per year
                       yearly running = 40 response + 90 residual = 130
Cumulative cost (spend + residual loss), years 1 / 2 / 3:
  A: 250 + 30n = 280 / 310 / 340
  B: 100 + 130n = 230 / 360 / 490
  Do nothing: 300n = 300 / 600 / 900

In year 1 B is cheaper (230 against 280; B's 230 is the one-time 100 plus its first year of 130) and both beat doing nothing. A overtakes B at (250 - 100) / (130 - 30) = 1.5 years. Over 3 years A saves 560 against doing nothing and B saves 410. Intuition: A costs more up front and removes more loss every year.

Recommendation. A, if the risk is expected to persist beyond about 18 months, as a likely leak will. Choose B if the horizon is under 18 months or if budget this year is the binding limit.

What would change my call. If the preventive control is really about 73% effective or lower, B wins over three years. Here e is A's true effectiveness. A's three-year cost is 250 + 300 x 3 x (1 - e), and B's is 490, so set them equal: 900 x (1 - e) = 240, so 1 - e = 0.267 and e = 0.733. If the $250k recurs every year, A costs 250 + 30 = 280 a year, while B costs 230 in year 1 and 130 in each later year (its 100 is paid once), so B wins in every year. Both can be layered: detection still catches what a preventive control misses.

Non-financial factors

• Prevention stops a data leak before it happens; detection starts after the data is gone, and regulatory or customer harm does not respond to a 70% cost reduction.
• Time to implement and who must maintain the control; false-positive friction for users.
• Whether 90% and 70% are tested or vendor claims.
• Regulatory duties, contract commitments, and reputation.

Scaling the same test: a $3M preventive control versus a $1M reserve (money set aside to absorb a loss), for a risk with expected loss of $2.5M per year. A $3M one-time control that removes 90% saves $2.25M per year and pays back in about 1.3 years (3 / 2.25). A $1M reserve removes no expected loss; it funds 40% of one year's expected loss (1 / 2.5), so it is risk acceptance with a buffer (you knowingly live with the risk but can pay for part of it), not mitigation. It suits risks that are rare but not preventable.

Pitfall. Comparing a one-time cost with an annual one without saying so.

Direct answer. Before signing, I would run a short, evidence-led risk discovery: map what the monolith really does, hold one structured workshop, prove the scariest unknowns with small spikes (time-boxed technical experiments), then price and gate the plan on what is found. Afterwards I would hand the team a ranked risk register with owners and triggers, a wave plan ordered by risk, go/no-go gates, rollback plans and a contingency reserve sized from the top risks.

Terms used here: cutover is the moment live traffic switches from the old system to the new; a wave is one batch of services migrated together; a go/no-go gate is a checkpoint where named people decide to proceed or stop based on a checklist; a rollback plan is the tested way back to the old system; a contingency reserve is spare time or budget held for expected slips; a trigger is the signal that says a risk is materialising.

1. Gather evidence first.

• Dependency and data mapping: batch jobs, file drops, scheduled tasks and integrations nobody documented (the usual source of surprises).
• Traffic and data profile: peak load, database size, cutover window the business can tolerate.
• Constraints: licences tied to hardware, compliance and data-residency rules, team skills, client decision speed.
• History: what went wrong on the client's earlier projects.

2. Run a 180-minute workshop. Participants: client engineering lead, operations or SRE (site reliability engineering), database owner, security, product owner, finance or procurement, our delivery lead and architect.

That is six segments summing to 180 minutes.

Scales used in the scoring segment (illustrative): likelihood is Low under 20%, Medium 20% to 50%, High over 50% within the nine months. Impact 1 is under a week of slip, 2 is 1 to 2 weeks, 3 is 3 to 5 weeks, 4 is 6 to 10 weeks or a customer-visible outage, 5 is a lost deadline or a data or security event. In a real session the probabilities would come from owners giving a low, likely and high estimate, checked against what happened on the client's earlier projects; the reserve table below uses illustrative values, not measurements.

3. Prove the top unknowns. Rehearse a data migration on a copy, load-test one extracted service, and confirm the hardest integration works end to end. A result changes the price or scope; a guess does not.

4. Hand-off package.

• Register: cause, event, consequence, score, owner, trigger date or signal, response. One row (illustrative): cause: undocumented nightly batch jobs read the monolith's tables; event: a job breaks after cutover; consequence: invoices stop for days; likelihood 40% (Medium), impact 4; owner: client database lead; trigger: any new dependency found in the discovery map; response: rehearse the job on the migrated copy before wave 3.
• Assumptions log with who validates each and by when.
• Wave plan: low-risk services first to learn; the riskiest cutover after rehearsal, each wave with a rollback plan and a go/no-go checklist. Illustrative: wave 1 is the internal reporting service (rollback: switch DNS back within an hour); wave 2 is the customer portal; wave 3 is the billing database cutover, allowed to proceed only if the rehearsal finished inside the agreed cutover window and the parallel run showed matching totals.
• Contingency reserve from expected slip (illustrative probabilities):

The five expected values sum to 6.35 weeks, about 16% of a 39-week plan (nine months is roughly 39 weeks). Summing assumes the risks are independent, so I would present the figure as a basis for a reserve, not a promise. A reserve equal to the expected value is also only a coin-flip level of protection: simulating these five risks as independent events gives about a 57% chance that the total slip is at or below 6.35 weeks, while a reserve of about 10 weeks covers roughly 80% of outcomes and about 13 weeks roughly 90% (the worst case, all five landing, is 26 weeks). So I would agree the confidence level with the client (for example 80%) and size the reserve from that percentile rather than from the average.

Inheriting a project with no register. In the first 48 hours: day one, read the plan and interview five people, asking what worries them and what they are assuming (for example, "who would be paged if the nightly job failed?"); draft a starter register by the end of the day. Day two, validate it with the sponsor, assign owners, set a review cadence and book the first spikes.

Market entry or alliance. The same cycle applies: swap the evidence (local legal and regulatory review, partner operations) and the participants (partner and channel owners), and run the pre-mortem on why the entry or alliance failed. For example, for entering a new country, a pre-mortem note might read "we could not sign local customers because data had to stay in-country", which becomes a register row with a legal owner and a spike to confirm hosting options.

Pitfalls. A register nobody links to the schedule. Risks without owners. Treating a signed contract as the end of risk discovery.

Direct answer. I would compare the options on expected annual downtime cost against the full extra cost of multi-region, show the sensitivity to the uncertain inputs, and recommend the option that survives the plausible range. With the illustrative inputs below, single-region launch wins by a wide margin, with a defined trigger to revisit.

Terms. A managed service is infrastructure the cloud provider runs and patches for you (for example a managed database). Single-region means everything runs in one geographic cloud region, so a regional outage (a failure that takes the whole region down) stops the service. Multi-region runs in two or more regions; here it needs custom orchestration, meaning our own code and runbooks for traffic routing, data replication and failover rather than a provider feature, which we must build, test and staff.

1. Downtime risk and revenue exposure (illustrative assumptions, not provider SLAs)
I put everything on one basis, dollars per year, and start with revenue because it is the easiest number to defend. Revenue $20M per year, which is $20,000,000 / 8,760 hours = about $2,283 per hour. Assume single-region availability 99.9% and multi-region 99.99% (availability is the fraction of time the service works).

Avoided exposure is about $18,000 per year (7.88 hours).

2. Cost and effort. Assume multi-region adds $180,000 per year in infrastructure and operations and 4 engineer-months of build (about $60,000, amortised over 2 years = $30,000). Total extra = $210,000 per year, versus $18,000 avoided: not close. Engineering effort is also a delay in launch and permanent complexity (custom orchestration needs its own failure testing and on-call knowledge).

3. Break-even. Multi-region pays on revenue alone only if revenue per hour is about $210,000 / 7.88 = $26,600, roughly $233M per year at these assumptions. I start from revenue alone because it is the one term both sides can check; then I list what is not in the arithmetic and could move the answer: contractual SLA penalties (credits we owe customers for missing an availability promise), regulatory data residency (rules requiring data to stay in or be available from certain countries) or resilience requirements, reputational cost of a long outage, and the correlated-failure risk that 99.9% hides. Correlated failure means the two regions do not fail independently: a bad deploy, a faulty configuration push or a shared DNS, identity or orchestration layer can take both down at once, so the 99.99% figure is optimistic unless the failover path is tested.

4. Defending under uncertainty. I vary the three biggest unknowns (availability gap, cost, revenue per hour) and show that the recommendation holds across the range, or state exactly where it flips. I also separate reversible from irreversible: single-region on a managed service can add a cross-region backup or a standby later; choosing custom multi-region first cannot be undone cheaply.

Recommendation. Launch single-region, with tested backups restorable in another region, an agreed recovery time objective (RTO, how long we accept being down) and recovery point objective (RPO, how much data we accept losing), and a revisit trigger such as revenue per hour crossing the break-even, an enterprise contract demanding it, or an actual regional outage.

Pitfall. Treating the availability figures as facts: they are assumptions to challenge.

Direct answer. In discovery I would present eight risks, scored on a shared scale so the client can see the order, each with a mitigation that fits in a proposal. I would sell a discovery phase as the first mitigation, because most of these risks are unknowns that discovery turns into knowns.

Scoring method (so the numbers mean something). Likelihood (chance the risk happens on a migration like this): High = 3 (more than about 1 in 2), Medium = 2 (roughly 1 in 5 to 1 in 2), Low = 1 (under about 1 in 5). Impact: 1 = no visible effect on date or customers; 2 = under a week of slip, absorbed by the team; 3 = a slip of 1 to 3 weeks, or a cost overrun the client must approve; 4 = a slip of more than 3 weeks up to the whole deadline being threatened, or a customer-visible outage lasting hours; 5 = the 9-month deadline is missed outright, or data loss or a serious security event. Priority = likelihood points x impact, maximum 15. These are suggestions to test with the client, not measurements.

Terms in the table. A landing zone is the pre-built, secured cloud foundation (accounts, networks, logging) that workloads move into. Guardrails are automatic rules that block unsafe settings. Least-privilege IAM means identity and access management where each person or service gets only the permissions it needs. Cutover is the moment live traffic switches from the old system to the new one, and the final sync copies the last changes across just before it. A parallel run operates old and new side by side and compares results. Right-sizing means resizing cloud resources to what the workload actually uses. A go/no-go is a checklist meeting where named people decide to proceed or stop.

One row traced. Risk 3, data migration and cutover downtime: likelihood Medium (2) because data volume is usually known only roughly before discovery; impact 5 because a failed or overlong cutover means a missed deadline and possible data loss; priority 2 x 5 = 10. Risk 4, skills gap: likelihood High (3) because the client team has run on-prem only; impact 3 because it slows operations by weeks but rarely breaks the date alone; priority 3 x 3 = 9.

Sorted by priority this reads 12, 10, 10, 9, 9, 8, 8, 6; the table order matches. Ties (10, 10 and 9, 9, and 8, 8) are broken by whether the risk blocks the date.

How I would present it. Lead with risks 1 to 3, because they threaten the 9-month date and the client's data. Flag that a 9-month plan for a monolith is only credible if discovery finds few hidden dependencies: that is the assumption the whole timeline rests on, and I would say it plainly even in a sales setting, because an underestimated plan costs the client and the relationship later. Also say that the scores will be re-run after discovery.

Pitfalls. Listing generic cloud risks with no link to this client; scoring everything high; hiding the cost and skills risks because they are awkward in a sale; and promising mitigations (such as zero downtime) the architecture cannot deliver.

Direct answer. I would run a structured failure-mode review, known as FMEA (failure mode and effects analysis): list how each part can fail, what the effect is, how likely and how detectable it is, and rank them to decide where to act. The output is a ranked list of failure modes, each with a mitigation and a named owner.

1. Find the failure modes. Draw the payment flow (checkout, payment service, fraud check, payment provider, ledger, settlement). For each step ask: what if it fails, is slow, returns wrong data, duplicates a message, or is called twice? Include dependencies, config and deploy changes, and people-driven steps. Gather the on-call and service owners together, and use past incidents as prompts.

2. Score on agreed scales (1 to 10, defined up front)

• Severity: how bad the effect is (10 = customers lose money or data).
• Occurrence: how likely the cause.
• Detection: how likely we are to catch it before customers do. This scale runs backwards: 10 = we would not notice, 1 = we would certainly catch it.
• RPN (risk priority number) = Severity x Occurrence x Detection. Illustrative scores:

3. Limits of RPN. The scores are ordinal ranks: numbers that only say "higher or lower", not "how much", so a 10 is not "twice" a 5 and multiplying them is shaky. The ledger loss (severity 10) ranks last at 40 while a slow page scores 100. So I add a rule: any severity 9 or above gets review regardless of RPN. Applied to the table, that pulls in the duplicate charge (S 9, already first) and the ledger write lost during failover (S 10, otherwise last), and no other row. The automotive AIAG and VDA FMEA handbook (2019) addresses the same weakness differently: it replaces RPN with Action Priority (AP), a lookup table that maps each combination of severity, occurrence and detection to High, Medium or Low priority for action, giving severity heavy weight so that a severe failure is not buried by multiplying three ordinal ratings together as RPN does. (AP is still a function of all three ratings, so a severe failure mode with a very low occurrence rating can still land in a low priority band; the severity override rule above is a separate safeguard.)

4. One failure mode end to end: duplicate charge. Cause: the payment service times out waiting for the provider, the client retries, and the provider has already charged. Detection today: only customer complaints (D=6). Mitigations: idempotency keys (a unique request ID so the provider ignores repeats), a reconciliation job (a scheduled check that compares every charge at the provider with our orders and flags mismatches), and an alert on duplicate-key rates. Owner: payments tech lead, due before launch. Rescored: S 9, O 2, D 3, RPN 9 x 2 x 3 = 54.

5. Turn into owned work. Each row gets an owner, a due date before launch, a verification (a test that injects the timeout), and a re-score after the fix. Unfixed items are accepted explicitly by a named executive. For failover, the mitigation row would state what happens when the primary database is switched to its standby (failover) and test that no ledger write is lost.