Incident Communication and Stakeholder Management Questions

Direct answer

Open by taking the report seriously and stating the next concrete step (no defending), explain the cause in plain words tied to the client's symptom, and commit to three things: a fix or workaround with a time, a named update cadence, and a prevention change. Do not argue about whether the client "should have" handled the change.

Structured elaboration

Opening response (first 30 seconds): thank them, confirm impact, say what I am doing right now, ask for what I need.

Terms for a Product Manager. A compatibility shim is a small piece of glue code that keeps the old behaviour working while the new one exists, for example returning both the old and new field. Versioning means publishing changes under a numbered version (v1, v2) so existing clients keep using the one they built against. A deprecation notice is an advance, dated warning that something will be removed. A request ID is a unique tag on each call that lets us find that exact request in our logs. A rollback is reverting the release to the previous version.

Explaining the cause: symptom (what their integration saw), trigger (our change), why it broke (the contract they depended on shifted), in one short paragraph. An API is the interface other software calls, and a breaking change is one that makes existing callers fail.

Commitments: (1) mitigation now: restore old behaviour behind a compatibility shim, or a rollback; (2) a time for that and an update time; (3) a permanent fix and a prevention step (versioning, deprecation notice, contract tests, which are automated checks that our responses still match what consumers expect).

Worked example (illustrative role-play)

Scenario: we renamed the response field customer_id to account_id in a release. The client's integration reads customer_id and now fails.

PM: "The client's integration broke after your API change. They are furious."

Me (opening): "Thanks for flagging it. I can see how that hurts them. First step, I am checking whether reverting the field name is the fastest safe fix. Can you get me their error message or a failing request ID so I can confirm it is the same issue?"

Cause: "Yesterday's release renamed customer_id to account_id in the order response. Their integration reads the old name, so it now gets nothing. That is a breaking change and it was not announced or versioned. That is on us."

Commitments: "By 3pm I will return customer_id alongside account_id so their integration works unchanged, and I will confirm with you. I will update you at 3pm and at end of day. This week we add a contract test that fails the build on renamed fields, and any future rename gets a deprecation notice with a date. I will send you a short written summary you can forward to the client."

If the client is still angry. PM: "They say a same-day fix is not good enough and want to know why this was not caught." Me: "That is fair. Nobody should have found this before we did. Today's fix stops the breakage; the reason it slipped through is that we had no automated check on field names, and that is what we are adding this week. I will not ask them to be happy about it. I will ask for a call after the 3pm fix so I can confirm their integration is healthy, and I will send the request-level detail of what we saw."

Trade-offs and pitfalls

• "Technically the docs said it could change" may be true and is still the wrong opening.
• Do not promise a permanent fix time before you know it; promise the mitigation and the next update.
• Do not hide that it was our change, and do not blame the client's code.
• What would change the call: if reverting breaks other clients who already migrated, keep both fields (the shim) instead of reverting.

Direct answer

Separate the two conversations from the fix. Executives get a steady, honest status and a clear view of exposure. Customers get an acknowledgement, a named contact, and a rhythm of updates. Credit or compensation (money or service time given back for missing the promised uptime) is discussed through the contract process (the steps the customer agreement lays out for claiming it), after the facts are measured, and never improvised mid-incident by the TPM (Technical Product Manager).

Executive conversation

• One update template: state, impact, next step, next update time. Send it on a fixed cadence so they stop calling engineers.
• Flag exposure: which customers have SLA (service-level agreement, the contractual uptime promise) commitments, and the rough credit liability (the total credits you may owe), labelled as an estimate.
• Ask for decisions only they can make: engineer priorities, approval of customer messages, and who speaks to the largest accounts.

Customer conversation

• A senior, named contact per high-profile account, one call early, then written updates on the promised schedule.
• Say what is known, what is not, and when you will speak next. Do not guess an ETA for the fix.
• If asked about credit: "We will measure the downtime against our agreement and follow the contractual process. We will come back to you on that once service is stable." Do not debate liability during the incident.

How SLA credit is calculated

Uptime SLAs are usually monthly. A 99.9% promise in a 30-day month allows (30 days times 24 hours times 60 minutes = 43,200 minutes in the month):

$43{,}200 \times 0.001 = 43.2 \text{ minutes}$

Suppose the outage lasted 120 minutes. Availability is:

$1 - \frac{120}{43{,}200} \approx 99.72\%$

That is below 99.9%, so a credit tier applies. A credit tier is a table in the contract mapping availability to a refund of that month's fee. For example (illustrative, not from any real contract): below 99.9% but at least 99.0% earns a 10% credit; below 99.0% earns 25%. At 99.72% the customer falls in the first band, so on a $5,000 monthly fee the credit is 10% times $5,000 = $500. Tier percentages differ by contract, so quote them from the actual agreement, not from memory. Credits often require a customer claim, but for top accounts many companies apply them automatically as goodwill.

Recommendation

Apply the contractual credit proactively for high-profile customers once the numbers are confirmed. It costs little against the relationship, and the decision belongs to finance and legal, with the TPM preparing the numbers. Offer anything beyond contract only with executive approval, and after the fix.

Trade-offs and pitfalls

• Promising credit early can be read as admitting fault and complicate legal position.
• Refusing to discuss credit at all reads as stonewalling. Acknowledge the topic and give a timeline.
• The TPM should not be both incident commander (the person directing the technical response) and communicator; delegate the fix so someone owns the messages.

Direct answer

Stop the error, quantify who was hit and by how much, then tell every affected customer proactively and specifically, before they discover it. Refund automatically, add a goodwill gesture (a credit beyond the refund, given to acknowledge the inconvenience), reach out personally to those who churned, and keep a visible weekly rhythm of updates until you can show what controls now prevent a repeat.

Sequence over the following weeks

• Day 0: fix and freeze the pricing logic, and confirm the exact scope (which customers, which period, over- or under-charged).
• Day 1: a public acknowledgement plus a personal email to each affected customer with their own amounts. Finance approves the remedy; legal reviews wording.
• Days 2 to 7: refunds issued automatically (no claim form), a goodwill credit, support briefed with a single script and a dedicated queue. Personal outreach to customers who cancelled, with an easy way back.
• Weeks 2 to 4: a short public write-up of what happened and which controls were added (pricing changes now need review and a reconciliation check before billing runs; a reconciliation check compares what the system billed against what the price list says it should have billed, and flags any mismatch). Then report results: refunds completed, complaints trending down.

Remediation and compensation principles

• Automatic, not by request. Making people ask compounds the harm.
• Exact numbers per customer, so the message is credible.
• Undercharged customers: normally do not claw back (ask them to pay the shortfall after the fact); decide with finance and legal, and say so plainly.
• Goodwill should be proportionate and simple, not a maze of tiers.

Rebuilding trust

Trust returns through repeated proof, not a single message. Track refund completion rate (refunds paid divided by refunds owed, for example 1,140 of 1,200 is 95%), complaint volume, and churn (customers who cancel) among affected customers versus a baseline of similar unaffected ones, meaning customers of the same plan and age who were not billed wrongly. If affected customers cancel at 4% a month and the baseline is 1%, the error is still costing you. Report those honestly to leadership.

Worked example (illustrative numbers)

A bug billed 1,200 customers $49 instead of $39 for one month.

• Overcharge per customer = $49 minus $39 = $10.
• Total refund = 1,200 times $10 = $12,000.
• Goodwill credit of $15 each = 1,200 times $15 = $18,000.
• Total cost of remedy = $12,000 plus $18,000 = $30,000.

Now compare it with lifetime value (LTV, the total revenue a customer brings before leaving). Suppose a customer pays $39 a month and stays 24 months on average: LTV = $39 times 24 = $936. If just 5% of the 1,200 churn because of the error, that is 60 customers, and 60 times $936 = $56,160 of lost revenue. The $30,000 remedy is smaller than that loss, which is the case you make to finance.

Trade-offs and pitfalls

• Compensating generously risks setting a precedent; compensating stingily amplifies public complaints. Tie the amount to actual harm plus a modest gesture.
• Vague apologies ("some customers may have been affected") read as evasion; specific ones read as ownership.
• Announcing a fix without a control (review or reconciliation) invites the same story next quarter.

Direct answer

Open by owning the call and lowering the temperature, not defending. Then give them a clear split: what we know, what we do not, and when they will hear next. Commit only to things you control: update times, a named contact, and a written summary. Do not guess a fix time or a cause.

Structure of the call

1. Open (first 30 seconds): introduce yourself, say you own their case, apologise for the impact, and let them speak. Do not interrupt or explain in the first minute.
2. Acknowledge specifics: repeat back their impact ("your users cannot sign in and your support lines are full"), so they know you heard the business problem.
3. What we know: confirmed facts only (start time, that it is broad, that the team is working on it, what is confirmed for their services).
4. What we do not know: be plain that the root cause and time to fix are unconfirmed, and why you will not guess.
5. Commitments: update every 30 minutes (even if "no change"), a direct number, a written summary within a day after recovery, and an initial root-cause report within about five business days.
6. Ask what they need: their most critical systems, workarounds they can use, and help drafting a message for their own users.

Example script (illustrative)

"Thank you for calling. I'm Priya, I'm responsible for your account through this. I'm sorry: I know your teams and customers are hurting. What I know: the problem began at 09:40, it is global, and our engineers are working on it now. What I do not know: the confirmed cause or how long recovery will take, and I would rather not guess. I will call you at 10:30 with an update whether or not it has changed. This is my direct number. Which of your services are most critical, so I can push those to the front?"

What not to say

• "It should be back soon." (a guess that becomes a broken promise)
• "It is not just you." (comfort for us, not for them)
• Any root cause that is not confirmed, or any blame on a partner or on their configuration.
• Jargon and internal ticket numbers.

Handling anger

Let them vent, keep your voice slow, avoid "calm down" or "as I said," and turn frustration into a concrete next step. If they escalate to a threat (contract, exit), acknowledge it and route it to the account owner and legal; do not negotiate on the call.

Trade-offs and pitfalls

• Saying "I don't know" feels weak, yet it is the honest position and it protects credibility later.
• Committing to more than you can deliver is worse than committing to less: a missed update time hurts more than a slow fix.
• Skipping the follow-up call breaks the one commitment they can hold you to.

Direct answer

A good public post-incident summary tells customers, in plain words: what went wrong, who was affected and how, what we did to fix it, and what we are changing so it does not recur. For a day of wrong results the priority is impact honesty: say exactly what was wrong, for whom and what they should check or redo. Keep it plain by leading with effects, and stay transparent by stating the unknowns and owning the cause without blame or spin.

Draft (illustrative, placeholders in brackets)

Title: Incorrect results in [feature], [date] 08:00 to [date+1] 08:00 UTC

Summary
For about 24 hours, [feature] showed incorrect [figures] for some customers.
The problem is fixed and results are now correct.

What happened
A change we released on [date] altered how [figures] were calculated.
It passed our tests but behaved differently on real data. We noticed when
customers reported mismatches and rolled the change back.

Who was affected
Roughly [X percent] of accounts that ran [feature] during the window. If
you did, your affected reports are listed in [link]. Results outside that
window were not affected.

What you may need to do
If you used those [figures] for decisions or exports, please re-run them.
We have already corrected stored results and will email affected accounts.

What we are changing
- Tests using production-like data before release.
- An automatic check that flags results differing from the prior day.
- A faster path from a customer report to our on-call team.

We are sorry. Questions: [contact].

Filled-in version of the same summary (illustrative facts)

Title: Incorrect totals in Usage Reports, 3 March 08:00 to 4 March 08:00 UTC

Summary
For about 24 hours, Usage Reports showed totals that were too low for some
customers. The problem is fixed and totals are now correct.

Who was affected
About 1 in 10 accounts that ran a report in that window. Affected reports
are listed at status.example.com/incident-0303.

What you may need to do
If you used those totals in a billing review or export, please re-run them.

Why it is written this way

• Title and first line carry the whole message: impact, period and status.
• Impact section is concrete: who, what, when, how to find out if you were affected. Wrong-results incidents differ from outages because customers may have acted on bad data, so the "what you may need to do" section is essential.
• Cause in one plain paragraph: name the type of cause (a change that behaved differently on real data), skip internal names.
• Prevention items are specific and checkable, not "we will be more careful".
• Transparency without exposure: state what we know and what we are still checking ("we are reviewing whether any exports were sent to third parties"). Do not include customer names, security-sensitive detail or speculation.

Worked example of plain wording

Three internal terms, in plain words: a regression is a change that breaks something that used to work; an aggregation job is a scheduled program that adds up raw records into totals; a stale cache hit means the system reused an old saved answer instead of recalculating.

Internal: "A regression in the aggregation job's rounding logic produced a stale cache hit." Public: "A change to how we add up your figures gave wrong totals for some accounts." Same fact, one less barrier.

Trade-offs and pitfalls

• Vagueness ("some issues") looks evasive; over-precision that you cannot back up (an exact affected-customer number you have not verified) is worse. Use verified numbers or clear ranges.
• Passive blame-shifting ("a vendor problem") loses trust unless it is true and you still own the customer outcome.
• Have legal and support review before publishing, and align the summary with what support has already told customers.
• Publish soon after resolution; a late summary looks like hiding.