Communicating Data and Analytical Findings Questions

Direct answer

When your analysis only supports "likely" or "unlikely," give Legal calibrated language anchored to a stated range rather than either a false absolute or a vague hedge, since a numeric anchor is what lets Legal reason about risk precisely even when you can't give them certainty.

Structured elaboration

Calibrated phrasing examples: instead of "this will happen" or "we don't know," use "based on our analysis, this outcome is likely, meaning we estimate roughly a 70-80% chance, though we can't rule out the alternative"; instead of "this is safe," use "we assess this as low risk, under a 10% chance of the concerning outcome, based on [specific evidence]." Each phrasing pairs a plain-language likelihood word (likely, unlikely, low risk) with an explicit numeric anchor, which satisfies Legal's need for something precise enough to act on while staying honest about the remaining uncertainty. The same discipline extends to citing sources: when Legal wants citation-ready language, attach the specific data source or precedent behind each claim directly in the same sentence, rather than as a separate, easily-detached footnote, and quantify exposure estimates (like estimated legal exposure probabilities for a new feature) using the same anchored-likelihood-plus-source structure.

Worked example

"Based on our analysis of [specific data source], we assess it as likely, roughly 70-75% probability, that [outcome], though we cannot rule out [alternative] given [specific limitation in the evidence]. This estimate is grounded in [specific precedent or dataset], attached here for reference." This gives Legal both the calibrated language they need to write a defensible position and a clear, attached basis for where the number came from.

Trade-offs and pitfalls

The temptation under pressure from Legal for a definitive answer is to either round the calibrated estimate up into false certainty, or to retreat into unhelpful vagueness that provides no numeric anchor at all; both fail Legal's actual need, which is a precise-enough, honestly-caveated number they can build a defensible position around, not a binary yes or no they'd have to overstate on your behalf.

Direct answer

A confidence interval describes uncertainty about an AVERAGE or summary statistic; a prediction interval describes uncertainty about a single FUTURE observation, and the prediction interval is always wider because it carries both the uncertainty in the average and the natural spread of individual outcomes around it.

Structured elaboration

• Confidence interval: answers "how sure are we about the average value?" Computed from the standard error of the estimate (how much the average would vary if we resampled the data), narrowing as sample size grows.
• Prediction interval: answers "where will the NEXT individual data point likely fall?" Computed from the same average uncertainty PLUS the natural variability between individual observations, and it does not shrink to zero even with infinite data, because individuals will always vary.
  Use a confidence interval when the business question is about a population-level number (average order value across all customers); use a prediction interval when the question is about one specific case (how much will THIS customer likely spend next month).

Worked example

Suppose average order value is $50 with a 95% confidence interval of $48 to $52 (we're confident the TRUE average is in that narrow range). The 95% prediction interval for a single customer's next order might be $10 to $110, because individual orders vary far more than the average does. Reporting the $48-$52 range as if it told you what any one customer will spend would badly understate the real variability a single prediction carries.

Trade-offs and pitfalls

The most common mistake is reporting a confidence interval when the business question is actually about an individual case (a specific customer, a specific day), which makes the uncertainty look far smaller than it really is. When in doubt about which to show, ask whether the audience is deciding something about the AVERAGE (staffing based on typical daily volume) or about ONE instance (whether a specific big order this week is plausible); that determines which interval belongs on the slide.

Direct answer

A dual-axis line chart plotting two unrelated metrics (say, model accuracy and unrelated revenue) is a chart to avoid, since lining up two independent y-axes visually implies a relationship between the series that may not exist at all; a clearer alternative is two separate small charts, or one chart with a single shared, meaningful axis.

Structured elaboration

The problem with dual-axis charts specifically: the two axes' scales are chosen somewhat arbitrarily (often just to make both lines fit nicely on the same plot), so whether the lines appear to move together or apart is an artifact of that scaling choice, not a fact about the data. A viewer has no way to tell, just by looking, whether the visual correlation is real or an accident of how the axes were stretched.
A clearer alternative for showing two related metrics, like model accuracy and business KPI over time, is either two small charts stacked vertically sharing the same x-axis (time), so the reader can compare timing of movements without a misleading implied scale relationship, or, if a single relationship IS the point, a scatter plot of one metric against the other with time encoded as color, which shows the actual relationship rather than an axis-scaling illusion.

Worked example

A slide shows model AUC (left axis, 0.70-0.80) and weekly revenue (right axis, $80k-$120k) as two lines that appear to move in lockstep. Rescaled independently (revenue axis stretched or compressed), the same two lines could be made to look correlated or uncorrelated at will, which is exactly why the chart is misleading regardless of which version happens to be shown. Two stacked charts, AUC over time on top and revenue over time below, sharing only the x-axis, let a viewer judge for themselves whether the timing lines up, without the artificial visual pull of a dual y-axis.

Trade-offs and pitfalls

The dual-axis chart is seductive precisely because it looks efficient, one chart instead of two, and can look impressively "connected." The cost is that it invites exactly the kind of correlation-implies-causation reasoning that a careful analyst should be actively working against elsewhere in the same presentation. When a real relationship between two metrics needs to be shown, prefer a chart type where the relationship is a direct visual encoding, like a scatter plot, rather than an artifact of two independently-chosen scales.

Direct answer

A dual-audience audit appendix works by giving non-technical auditors a plain-language summary of each key claim with a direct pointer to its evidence, while technical auditors get the actual artifacts (code, data lineage, test results) referenced from that same summary, so both audiences navigate the same document without either being shortchanged.

Structured elaboration

Outline: (1) a claims summary section, one plain-language sentence per key claim with a direct reference to the supporting artifact number; (2) a methods section describing, in prose, what was done and why, written for someone who wants to understand the approach without reading code; (3) the technical artifacts themselves, code snippets, a data dictionary, and test results, organized so a technical auditor can trace from any claim in the summary straight to its evidence; (4) a robustness-checks section specifically, showing the checks run against the headline result (for a revenue-uplift finding, for example: a check with an alternative time window, a check excluding outliers, and a check against a held-out sample), each summarized visually on a single slide or page rather than left buried in an appendix.
This same shape scales up to reproducible board-level material or investor-facing appendices: a board presentation's reproducible appendix should let a technical reviewer re-run key analyses by pointing to versioned code and data snapshots, referenced directly during the talk rather than left as a static, disconnected attachment; an investor-facing limitations-and-sensitivities appendix should list the top risk items (customer concentration, churn sensitivity, pricing elasticity) each quantified or qualified in investor-friendly language with a one-sentence magnitude, not a vague caveat.

Worked example

Claims summary: "Claim 3: the pricing change increased average order value by 4%, see Artifact B (regression output) and Artifact C (robustness check excluding top-1% outliers, which shows a similar 3.8% effect)." A non-technical auditor reads the claim and its plain-language support; a technical auditor follows the reference directly to the regression output and robustness check code.

Trade-offs and pitfalls

The main risk is writing a claims summary vague enough that a non-technical auditor can't actually verify anything from it, defeating the purpose of a dual-audience document; each claim should be specific enough that both audiences could, in principle, check whether the artifact actually supports it, even if only the technical auditor does so in practice.

Direct answer

Three edits usually fix a confusing model-explanation diagram: reduce the number of distinct visual elements to only what's load-bearing for the point being made, add a legend that defines every symbol used, and reorder the layout to follow the same left-to-right or top-to-bottom flow the audience will read it in.

Structured elaboration

• Reduce elements. A diagram trying to show every input, every layer, and every output at once usually shows none of them clearly; cut anything not needed for THIS particular explanation, and note that a fuller diagram can live in an appendix for those who want it.
• Add or improve the legend. Any shape, color, or arrow style used more than once needs to appear in a legend; a diagram that relies on the presenter to explain what a dotted line means has already failed for anyone reading it later.
• Reorder for reading direction. Data flow should follow the audience's natural reading path; a diagram that requires the eye to jump around to follow the sequence adds cognitive load that has nothing to do with the model itself.
  Each of these matters for BOTH technical and non-technical viewers, but for different reasons: a technical viewer is slowed down by clutter and ambiguous notation, while a non-technical viewer may abandon an unclear diagram entirely rather than work to parse it.

Worked example

A model-architecture diagram with 15 boxes, mixed arrow styles, and no legend, laid out in a circular pattern, is simplified to 6 boxes representing the stages that matter for the specific point being made, a single consistent arrow style with a legend defining "data flow" versus "feedback signal," and a strict left-to-right layout matching how the presenter will narrate it.

Trade-offs and pitfalls

The instinct to keep a comprehensive diagram "just in case someone asks about a specific component" is understandable but usually backfires; a diagram trying to serve every possible question ends up answering none of them clearly in the room. Better to have the simplified version for the presentation and the comprehensive version as a named backup slide, so the comprehensiveness is available without costing the room its clarity.