Client and Customer-Facing Communication Questions

Direct answer

Use a story that shows four things: you took the complaint seriously and reproduced it, you told the customer what you knew and did not know at each stage, you found a specific root cause, and you changed something so it cannot recur silently. The story below is an illustrative shape, a composite built to show the structure. Replace the details with your own real example, and use your own real numbers; do not copy these placeholders as if they were results.

Story skeleton (STAR: situation, task, action, result)

Situation. I worked on a scoring model that ranked a company's business customers by the risk of cancelling their subscription (a "churn risk" score; churn means customers leaving), used by the customer's account managers each Monday. One account manager told us the model was ranking clearly healthy accounts as high risk, and she was about to stop using it.

Task. Find out whether she was right, keep her informed, and make sure it could not happen again quietly.

Action, investigation.

1. I asked her for three specific accounts and what she expected, instead of debating the model in general. Three concrete cases can be reproduced; a general doubt cannot.
2. I checked the inputs for those accounts, not the model first. The "days since last login" value was extremely large for all three, though they were active. Tracing back, an upstream change to single sign-on (SSO, one login for many applications) had made the login-time field empty for those users, and our pipeline (the automated steps that prepare data before the model sees it) replaced empty with zero, which the model read as "has not logged in for years". Why: the login field is a date, and a stored zero means 1 January 1970, the starting point computers count dates from, so "days since last login" came out at over 20,000 days, more than 55 years, for a customer who logged in yesterday.
3. I measured the scope: how many of the accounts scored that week had that field empty (I reported it as N of M, with the exact figures from the data; in an illustrative case, 312 of 1,480 accounts scored that week, about 21%, had the field empty).

Action, communication along the way.

• Same day: "Thanks, I can see why it looks wrong. I'm checking the three accounts and will report back tomorrow."
• Next day: "What we know: the login-time input is empty for some accounts and is being read as inactive. What we do not know yet: how many accounts. Until it is fixed, please disregard the risk ranking for accounts with SSO."
• On fix: what was wrong in plain terms, which scores changed, and when corrected scores would reappear.

Action, prevention. I added a check that stops scoring and alerts us if the share of empty values in any input jumps above a set threshold (for example, alert if more than 2% of an input is empty when it normally runs below 0.5%). I changed the pipeline to treat empty as "unknown", not zero. And I added the three accounts as permanent test cases. I also wrote a short post-incident note shared with the customer. Its shape: "What happened: after a login change, some accounts had no login time and were scored as inactive. Who was affected: 312 accounts, scored on Monday 6 October. What we did: corrected the data and rescored; corrected rankings were in your inbox on Wednesday. What changes: we now stop scoring and alert us if inputs go missing."

Result. Corrected scores were delivered; the account manager kept using the model; the new check later caught a different input problem before any customer saw it. If you have measured outcomes in a real story (accounts corrected, time to resolution, number of recurrences), state them with their real values.

What separates a strong answer

• You started with the customer's concrete examples and reproduced the problem.
• You told them what you knew and did not know, and gave interim guidance, instead of waiting for a full answer.
• The fix addressed the cause (an input assumption) and the system (a check), not just the symptom.
• You did not blame the customer or the upstream team.

Pitfalls

Do not claim the customer was wrong without evidence. Do not describe only the technical fix; the communication is half of the question. Do not tell a story with no change at the end.

Direct answer

Treat the complaint as a reconciliation problem: agree on one concrete number that differs, make both calculations explicit, and walk from their figure to ours step by step until the gap is fully explained. Keep the client informed on a committed schedule, including days when there is no finding. Most gaps turn out to be a mix of definition differences and timing, with occasionally a real defect, and the investigation should separate those.

Step 1: agree on one concrete case (day 1)

Acknowledge within one business day and ask for: the report name, one specific metric, one date range, the number they have, and their method (usually a spreadsheet). Restate it back in writing: "We are comparing total orders for 1 to 7 October: your manual report shows 1,204, the dashboard shows 1,150." Narrowing to one metric and period turns a vague distrust into a testable question.

Step 2: write down both definitions

Ask side by side: what counts as an order (cancelled? test? refunded?), which date is used (placed, shipped, paid), which time zone, whether duplicates are removed, and how late-arriving data is handled. Then collect their source file and our query logic.

Step 3: reconcile with a bridge

Check: 31 + 15 + 8 = 54, and 1,204 - 54 = 1,150. The numbers are illustrative; the shape is real. Classify each cause as: definition difference (agree which is right), timing, our defect (fix it), or their manual error (explain gently, with evidence).

Step 4: keep them informed

• Day 1: acknowledgement and restated scope.
• Every 2 business days: a short update with what we ruled out, what we are testing and the next update date, even if the update is "nothing new".
• A shared document with the bridge so they can follow along and challenge it.
• On resolution: a one-page summary with cause, fix, and what changes.

Step 5: stop it recurring

Document metric definitions on the dashboard itself, add a tie-out check (an automated comparison against a trusted source), and set an alert when the difference exceeds a threshold. If they want their definition used instead of ours, change the dashboard and say so.

Pitfalls

Do not defend the dashboard before looking. Do not tell them their spreadsheet is wrong without a bridge that shows how. Do not go silent while you investigate, which does more damage than the discrepancy.

Direct answer

Lead with a calm, specific, bounded statement: what you checked, what you found (8% of records fail validation), what is and is not yet known about billing, what you are doing today, and when they will hear next. Panic comes from vagueness and surprise, not from bad news, so the cure is precision, a ranked plan and a date. Present billing-affecting failures first, because that is the part the client cares about.

Step 1: size the problem before the call

8% sounds large and says little. Break it down. Illustrative example: the audit covered 2,000,000 records, and 8% fail, which is 160,000 records (0.08 x 2,000,000 = 160,000). (Billing exposure here means the most money that could be wrongly billed; an invoice run is the scheduled batch job that generates and sends invoices.) Sort the failures by what they touch:

Check: 22,000 + 58,000 + 80,000 = 160,000. The headline becomes: "8% failed validation, but about 1.1% touch the amounts that drive billing" Here "touches billing" means the failing field feeds an invoice amount, and the 1.1% is category A only. Say how you know this (the validation rules, run on a given date) and how sure you are.

Step 2: prioritize by billing risk, reversibility and the next deadline

Fix in this order: A before B before C. Within A, put records on the next invoice run first. A wrong invoice that has already gone out is harder to undo than one still in draft, so the date of the next billing run drives urgency. Also estimate exposure honestly: if the average error on a Category A record is $4 (illustrative), the maximum exposure is $22,000 x $4 = $88,000, stated as a range with its assumption, not as a loss.

Step 3: propose the remediation timeline

The spans are reasoned, not arbitrary: Day 2 is two working days to trace 80,000 records' worth of rule failures back to a few causes; Day 5 gives time to correct 22,000 records, re-validate and get the client's approval of staged values; Day 10 allows a second batch of 58,000 simpler code fixes. Replace them with your own team's measured speed, and say they are estimates.

What I would say on the call

"We ran our routine data audit and found that 8% of records fail validation. I want to give you the picture straight away. About 1.1% of records touch amounts that affect billing, and I'll hold those out of the next invoice run until they are corrected, so no invoice is generated from a bad amount or rate. The rest are tax code and formatting issues we can fix on a normal timeline, and I'll check the tax code records against the invoice run and flag any that would produce a wrong or failed invoice. I'll send you the list and the plan today, and you'll hear from me again on Wednesday with the cause."

Avoid words like "corrupted", "failure" and "breach" when they do not apply, and avoid blame. Say "fail validation", which is accurate.

Trade-offs and pitfalls

• Do not wait for the full root cause before telling them. Disclose what you know with its confidence, and promise the rest by a date.
• Do not bury the billing risk. If you soften it, the client discovers it later and trust collapses.
• Do not promise a fix date you cannot verify. Offer a date for the cause and a range for the fix.
• If the next invoice run is tomorrow, the whole plan compresses: hold the run for category A, and brief the client's finance contact first.

Direct answer

Do not accept an unconditional guarantee of accuracy, because accuracy on future real-world data depends on things neither side fully controls: the quality and drift of the customer's data, how the labels were defined, and how the model is used. Instead, propose a measurable, evidence-based commitment: an agreed target on an agreed test set, reached through staged pilots, with remedies that are fair if it is missed. The customer gets a verifiable result and exit rights; you avoid an open-ended liability.

Why a flat guarantee is a bad idea

• Accuracy is not a single number. It depends on the metric (accuracy; precision, the share of the model's positive calls that are right; recall, the share of real positives it finds; or a cost-weighted score, which counts costly mistakes more than cheap ones), the class mix (the proportion of each category in the data) and the labels (the recorded correct answers). "95 percent accuracy" on a dataset that is 95 percent one class is achievable by always guessing that class.
• Data drift (the real-world data changing over time) can break a model that was good at delivery.
• Statistical uncertainty. A confidence interval is the range the true accuracy plausibly sits in, given that you only measured a sample. On a test set (examples with the correct answers already recorded) of 500 labelled examples where the model scores 90 percent, the 95 percent confidence interval is about plus or minus 2.6 points, from 1.96 x sqrt(0.9 x 0.1 / 500) = 0.0263 (this uses the normal approximation, which treats sampling error as a bell curve; 1.96 is the standard multiplier for 95 percent coverage; step by step, 0.9 x 0.1 / 500 = 0.00018, its square root is 0.0134, and 0.0134 x 1.96 = 0.0263). The plausible range is therefore about 87.4 to 92.6 percent. So 90 percent measured on 500 examples cannot prove the true accuracy exceeds 88 percent with confidence, let alone 92. A target must be paired with a test set big enough to verify it.

What I would say

"I can't promise a number on data neither of us has seen, but I can commit to something measurable. Let's agree the metric and a labelled test set that you own and we don't touch, set a target, and structure the work so you only pay in full once we hit it on that set. If we miss it, you have a defined remedy."

What I would put in the agreement

• Defined metric and definition of correct, with labelling rules written down.
• A frozen (never changed or trained on), jointly agreed test set, sampled to match production, large enough for the target (state the sample size and interval).
• A target and a minimum acceptable level, for example a target of 90 percent and a floor of 85 percent, both on the test set. The target is what we aim for; the floor is the minimum below which the remedies apply.
• Conditions: data quality and volume delivered as specified; the use case does not change; model performance after go-live is monitored.
• Remedies: an agreed period of additional tuning at our cost; fee credits tied to milestone payments; and a right to terminate the later phases without penalty if the floor is not met.
• No commitment to performance on data outside the agreed scope or after significant data drift, but an offer of a monitored support period with retraining.

Pilot phases that share risk

Each gate has a go or no-go decision by both parties on pre-agreed numbers, so either side can stop early at limited cost.

Payment illustration (invented figures): on a $300,000 project, the data audit is $30,000, Milestone 1 is $70,000, Milestone 2 is $90,000 and Milestone 3 is $110,000 (30 + 70 + 90 + 110 = 300). If the floor is missed in Phase 1 after the agreed tuning period, the customer stops and never owes the $200,000 in Milestones 2 and 3. If the floor is met but the target is missed in Phase 3, a fee credit of, say, 20 percent of Milestone 3 ($22,000) applies.

Trade-offs and pitfalls

• Accepting the guarantee to win the deal may create unbounded liability. Escalate to legal and leadership before agreeing anything numeric.
• Setting the target before seeing the data is a guess; make the data audit a gate and allow the target to be refined at the end of Phase 0 by mutual agreement.
• A single average hides failure: agree separate thresholds on the error type that matters (for example missed fraud versus false alarms).
• If the customer will not accept any shared structure, consider whether the deal is worth the exposure.