Communicating Data and Analytical Findings Questions

Direct answer

Proving that a pricing change, not something else, caused a revenue increase means ruling out the other plausible explanations one by one, not just showing the revenue and pricing timelines moved together.

Structured elaboration

Five analyses: (1) check whether the timing of the revenue change aligns precisely with the pricing change's rollout date, not a broader window that could hide other explanations; (2) segment the analysis to markets or customer groups where the pricing change did versus did not apply, if any exist, as a natural experiment; (3) check for confounding events in the same window (a marketing campaign, a competitor's exit, a seasonal pattern); (4) look at unit economics specifically, did average order value or transaction count change in the way the pricing logic would predict, rather than just total revenue; (5) if possible, run or reference a smaller controlled test of the pricing change in isolation.
Presenting this to a skeptical stakeholder: lead with the strongest piece of evidence (usually the natural experiment or controlled test, if one exists), then layer in the supporting checks, framing each ruled-out alternative explicitly rather than only presenting the confirming evidence.

Worked example

"The pricing change rolled out exactly two weeks before the revenue increase began, and where we had a natural comparison, one region delayed the rollout by a month, that region's revenue increase also lagged by the same month, which is strong evidence this isn't a seasonal or market-wide effect. We also checked: no major marketing campaign or competitor change coincided with the timing, and the revenue increase came specifically from higher average order value, consistent with the pricing mechanism, not from more transactions, which a different cause like a marketing push would more likely explain."

Trade-offs and pitfalls

The natural-experiment check (a region or segment where the change didn't apply, or applied later) is by far the strongest evidence available here, and its absence should be stated honestly rather than papered over with correlational timing alone; if no such comparison group exists, say so and calibrate confidence in the causal claim accordingly, rather than presenting timing correlation with the same confidence a natural experiment would warrant.

Direct answer

Presenting a result that contradicts an executive's strong prior belief works best when you've already built support BEFORE the room forms, so the meeting isn't the first place the finding surfaces, and when the deck leads with the process that generated the result rather than the conclusion alone.

Structured elaboration

Step-by-step approach: (1) before the main meeting, share the finding one-on-one with the executive whose belief it contradicts, giving them a private moment to react rather than a public one; (2) identify one or two other stakeholders whose support would carry weight in the room, and brief them beforehand too, so the presenter isn't the only voice defending the finding; (3) structure the deck to walk through the methodology and sensitivity checks BEFORE stating the conclusion, so the room has already evaluated the rigor before reacting to the surprising result itself; (4) surface uncertainty and sensitivity checks explicitly and proactively, framed as "here's what we stress-tested," rather than waiting to be asked; (5) end with a recommendation that acknowledges the discomfort of the finding while still being clear about what it implies.

Worked example

"Before presenting to the full group, I'd walk the VP through the finding privately first, giving them time to process it without an audience. In the room, I'd open with the methodology and the three sensitivity checks we ran, not the conclusion itself, so the group evaluates the rigor before reacting to the surprising result. Only then would I state the finding plainly: 'the data doesn't support the assumption we've been operating under; here's what it suggests instead, and here's what we checked to make sure this isn't an artifact.'"

Trade-offs and pitfalls

The instinct to soften a contradicting finding by hedging heavily in the room usually backfires, since a hedged presentation of solid analysis reads as LESS credible, not more diplomatic; the private pre-brief and the methodology-first structure do the diplomatic work, which frees the actual finding to be stated with appropriate confidence rather than buried in qualifiers.

Direct answer

Two variables moving together does not tell you which one, if either, is causing the other, or whether a third factor is driving both; the clearest way to show this is a case where the "obvious" causal story is provably wrong.

Structured elaboration

Two charts and captions that work well:

• Chart 1: ice cream sales and drowning incidents both rising over the summer months, captioned "these two rise and fall together" - purely descriptive, no causal claim.
• Chart 2: the same two series with a third line added, average daily temperature, captioned "both are driven by warm weather: more swimming AND more ice cream sales, not one causing the other" - revealing the shared driver (the classic confounder in this analogy).
  A short company policy that follows naturally: act directly on a correlated finding only when a plausible mechanism is already well understood and the cost of being wrong is low; require a controlled experiment before acting on any correlation that would drive a costly or hard-to-reverse decision.

Worked example

A live version of the same pattern: if a company observes that customers who use a certain feature have higher retention, it's tempting to conclude "the feature causes retention." Retention-prone customers (more engaged from the start) may simply be more likely to try new features too, making feature usage the ice cream and underlying engagement the temperature. Only a randomized rollout of the feature, comparing retention between an offered group and a not-offered group, can distinguish "the feature helps" from "engaged people use both features and retain better regardless."

Trade-offs and pitfalls

In the moment, when a stakeholder in a live meeting jumps straight from a correlation to a policy change, the useful move is not to lecture on causation abstractly but to ask one concrete question: "what's a plausible reason these two might move together WITHOUT one causing the other?" That question does the work the ice-cream analogy does, without requiring the audience to sit through a definitions lesson, and it usually surfaces the confounder the audience hadn't considered, which is what actually changes their mind.

Direct answer

A dual-axis line chart plotting two unrelated metrics (say, model accuracy and unrelated revenue) is a chart to avoid, since lining up two independent y-axes visually implies a relationship between the series that may not exist at all; a clearer alternative is two separate small charts, or one chart with a single shared, meaningful axis.

Structured elaboration

The problem with dual-axis charts specifically: the two axes' scales are chosen somewhat arbitrarily (often just to make both lines fit nicely on the same plot), so whether the lines appear to move together or apart is an artifact of that scaling choice, not a fact about the data. A viewer has no way to tell, just by looking, whether the visual correlation is real or an accident of how the axes were stretched.
A clearer alternative for showing two related metrics, like model accuracy and business KPI over time, is either two small charts stacked vertically sharing the same x-axis (time), so the reader can compare timing of movements without a misleading implied scale relationship, or, if a single relationship IS the point, a scatter plot of one metric against the other with time encoded as color, which shows the actual relationship rather than an axis-scaling illusion.

Worked example

A slide shows model AUC (left axis, 0.70-0.80) and weekly revenue (right axis, $80k-$120k) as two lines that appear to move in lockstep. Rescaled independently (revenue axis stretched or compressed), the same two lines could be made to look correlated or uncorrelated at will, which is exactly why the chart is misleading regardless of which version happens to be shown. Two stacked charts, AUC over time on top and revenue over time below, sharing only the x-axis, let a viewer judge for themselves whether the timing lines up, without the artificial visual pull of a dual y-axis.

Trade-offs and pitfalls

The dual-axis chart is seductive precisely because it looks efficient, one chart instead of two, and can look impressively "connected." The cost is that it invites exactly the kind of correlation-implies-causation reasoning that a careful analyst should be actively working against elsewhere in the same presentation. When a real relationship between two metrics needs to be shown, prefer a chart type where the relationship is a direct visual encoding, like a scatter plot, rather than an artifact of two independently-chosen scales.

Direct answer

Structure a negative-result story around three commitments: what actually happened (without softening the number), what you already ruled out or checked, and what happens next, since a room hearing bad news is really asking "is this handled" as much as "what happened."

Structured elaboration

A narrative arc for presenting a decline or a no-effect result:

1. State the result plainly, with the number, before any framing or context; burying the bad news in caveats reads as evasive.
2. Show you've already done basic diligence - ruled out a measurement error, checked whether it's isolated to a segment, confirmed it's not a seasonal pattern - so the room isn't left wondering if the result itself is even real.
3. Propose concrete next steps, ideally with a timeline, so the meeting ends with an action rather than just an acknowledgment.
   Balancing transparency with actionability means resisting two opposite failure modes: softening the number to manage the room's reaction (which erodes trust the moment anyone checks the real figure), and presenting the result with no path forward, which reads as simply delivering bad news rather than managing it.

Worked example

"Our new onboarding flow, which we expected to lift activation, actually showed no measurable difference. Before bringing this here, I checked three things: it's not a measurement issue (the instrumentation matches the old flow), it's not concentrated in one segment (the null result held across device types and acquisition channels), and it's not a power problem (the test ran long enough to detect an effect of this expected size had one existed). Given that, I don't think this specific flow is the lever; I'd recommend we don't roll it out further and instead look at [alternative hypothesis] next, with a first read in two weeks."

Trade-offs and pitfalls

A common instinct is to open with the diligence and mitigating context before the number itself, hoping to soften the landing; this usually backfires, since the audience senses the delay and often assumes the news is worse than it is. State the number first, then earn back the room's confidence with the diligence you've already done. A genuinely surprising or counterintuitive result (a metric that moved the WRONG way after a supposedly positive change) deserves the same structure, with extra emphasis on the validation steps, since a surprising result invites more scrutiny of whether it's real before anyone will act on it.