Problem Definition and Framing Questions

Direct answer

A high-quality problem statement names the specific target user, states the current measured condition, states the desired condition, gives a timeframe, and specifies the metric that will confirm the gap is closed. Leaving any one of these out is what turns "we should fix onboarding" into an unactionable slogan instead of a scoped, testable problem.

Structured elaboration

Walking through why each part earns its place:

• Target user: without a named segment, "users are dropping off" could mean everyone or one small cohort, and those two situations call for completely different investigations.
• Current vs desired state: this is the gap the rest of the process exists to close. Stated as two numbers (or two qualitative states with a clear boundary between them), it gives the team a shared definition of "done" before anyone proposes how to get there.
• Timeframe: both the window the current state was observed over, and the window given to close the gap. A statement with no fix-by date competes poorly for prioritization against work that has one.
• Metric: precise enough that two people computing it independently land on the same number. A metric like "engagement" without a computation rule invites disagreement later about whether the fix worked.

A statement that includes all four but is still too broad ("increase engagement, ever, by some amount") fails a different test: it can't be disproven, so nobody can ever say with confidence that it's been solved.

Worked example

For the scenario given: "Among users who received the design update, monthly active users (MAU, the count of distinct users active at least once in a 30-day window) dropped 12% in the 30 days after the update shipped. We want MAU for this cohort back to its pre-update baseline within 6 weeks, measured as the 30-day rolling MAU count for users on the updated design." That single sentence is specific enough that a different analyst, handed only the sentence, could pull the same number from the event logs.

The same components apply to a one-paragraph ML problem statement written for a product requirements document (PRD): target user, current versus desired state, and constraints carry over unchanged, with one addition, an ML problem statement should also name the evaluation metric and baseline the model will be judged against, since "success" for a model is otherwise undefined.

Trade-offs and pitfalls

The most common failure mode is a statement that smuggles in a cause or a solution ("MAU dropped because the new layout hides the main action, so we should revert it"), which shuts down investigation before it starts by presenting an unverified theory as settled fact. The second is treating this as a one-time exercise: the components matter because they are testable, not because they're a checklist to fill in once and never revisit if new evidence emerges during discovery.

Direct answer

Three biases that commonly distort problem framing: confirmation bias (favoring evidence that supports a theory you already hold), anchoring (over-weighting the first number or explanation you encounter), and availability bias (treating a vivid, memorable example as more representative than the actual data supports). Each has a specific, checkable mitigation, which is what makes naming them useful rather than just a caution.

Structured elaboration

Confirmation bias: in product analytics, this shows up as running a segmentation cut, seeing a pattern that matches your existing theory, and stopping there without checking whether an equally plausible alternative cut would explain the data just as well. Mitigation: before looking at the data, write down what pattern would DISPROVE your leading theory, not just what would confirm it, and specifically check for that pattern.

Anchoring: in product analytics, this shows up when the first stakeholder to weigh in ("I bet it's the pricing page") shapes the entire subsequent investigation, even after data arrives that's ambiguous between that explanation and others. Mitigation: generate your list of candidate hypotheses before hearing anyone's initial theory, or at minimum before looking at any data, so the list isn't anchored on whoever spoke first.

Availability bias: in product analytics, this shows up when one vivid customer complaint (a detailed, emotionally compelling support ticket) gets treated as representative of a broad pattern, when it may in fact be a single outlier. Mitigation: before acting on a vivid individual example, check its frequency against the full data; it may be real and worth acknowledging, but shouldn't set the scope of the response on its own.

Worked example

A stakeholder reports "customers hate the new pricing page," backed by one detailed, well-written complaint email. Applying the availability-bias mitigation: check the actual data first, support-ticket volume mentioning pricing, before and after the page changed, and the site's overall conversion rate at that step. If ticket volume mentioning pricing is flat and conversion at that step is unchanged, the vivid complaint, while real, isn't representative of a broader pattern, and treating it as one would misdirect the team's investigation toward a problem that isn't actually widespread.

Trade-offs and pitfalls

These mitigations take real discipline to apply consistently, particularly under time pressure, when writing down a falsifiable prediction before looking at data, or deliberately delaying hearing a senior stakeholder's initial theory, can feel like an unnecessary extra step; the value shows up specifically in the cases where the bias would otherwise have sent the investigation in the wrong direction, which by definition you can't always tell in advance. The other pitfall is using "that could just be confirmation bias" as a reflexive dismissal of a genuinely well-supported finding, rather than reserving the label for cases where the mitigation check (looking for disconfirming evidence, checking frequency against the full data) actually reveals a gap.

Direct answer

Ordered by risk reduction per unit of effort: (1) whatever data you already have access to, since it costs nothing new to check; (2) the single most consequential unknown, the one that would most change your recommendation if it came back differently; (3) whether anyone else has already solved or investigated something similar; (4) a quick, low-cost check on your leading hypothesis; (5) a broader validation pass only if the first four still leave real uncertainty.

Structured elaboration

1. Existing data first: before requesting anything new, check what's already available, dashboards, logs, past analyses, since this is the cheapest possible information and often resolves more uncertainty than expected before you've spent any new effort.

2. The single most consequential unknown: identify which specific unknown, if resolved, would most change your recommendation; investigating a fact that wouldn't actually change your decision either way is low-value regardless of how easy it is to check, so this step comes before anything else that isn't free.

3. Prior work: check whether the organization has already investigated something adjacent, an old analysis, a past experiment, a team that hit a similar question; this can save the entire remaining investigation if a close-enough answer already exists somewhere.

4. A quick check on the leading hypothesis: once you have a working theory from steps 1 to 3, a cheap, fast test (not a full experiment) that would meaningfully update your confidence, before committing more time.

5. Broader validation, only if still needed: a fuller investigation, additional data collection, a proper experiment, reserved for genuine remaining uncertainty after the first four steps, rather than the default starting point.

The ordering logic is strictly about return on a scarce resource, time: each step is ordered from lowest-cost, highest-value first, so that if you run out of time at any point, you've captured the highest-value information available at that cost level rather than partway through a more expensive step that hadn't yet paid off.

Worked example

Given limited time to recommend whether a reported "users can't find the export button" complaint is worth fixing: step 1, existing analytics already show the export feature's click rate over the past month, no new pull required; step 2, the most consequential unknown is whether this is a genuinely common complaint or one vocal user, so that's the priority to resolve, not (for instance) exactly how many pixels the button should move; step 3, a quick search finds a similar complaint was investigated eight months ago and traced to a specific onboarding cohort, which narrows where to look now; step 4, a fast query segmenting current click rate by that same cohort; step 5, a broader usability study only if the segmented data still leaves the picture unclear.

Trade-offs and pitfalls

This ordering assumes you can correctly judge, at step 2, which unknown is genuinely the most consequential; misjudging that (chasing an easy-to-check but low-impact question first) undermines the whole point of the ordering, so it's worth deliberately pausing on step 2 rather than rushing past it to the checks that feel more concrete. The other pitfall is skipping step 3 (checking for prior work) because searching for it feels like it takes effort, when it's often the fastest way to shortcut the rest of the list entirely.

Direct answer

State it as: "Among users who signed up by email in the past 14 days, onboarding completion fell from 80% to 60% after the new onboarding flow shipped; we want completion back above 80% within [N] weeks, measured as sessions that reach the 'setup complete' event." That single sentence names the target user, the current state, the desired state, a timeframe, and the metric that will tell you when the problem is solved.

Structured elaboration

A well-formed problem statement carries five load-bearing parts, and a statement missing any one of them invites the wrong kind of solution:

1. Target user - not "users" in general, but the specific segment the data is about (email sign-ups in the last 14 days, not all users). Naming the wrong segment sends the team investigating a population that was never affected.
2. Current state - the measured baseline, with enough specificity to be re-derived by someone else (80% completion, this cohort, this date range).
3. Desired state - what "fixed" looks like, ideally the prior baseline unless there is a reason to target something else.
4. Timeframe - both the window the drop was observed over and the window you're giving yourself to fix it. Without a fix-by timeframe, "reduce this" never gets prioritized against other work.
5. Metric - the exact signal you'll watch, defined precisely enough that two people computing it independently get the same number.

Two things a problem statement should NOT do: it should not name a cause ("completion fell because the new flow is confusing") and it should not name a solution ("we need to simplify onboarding"). Both foreclose investigation before it starts. The statement's job is to make the gap undeniable and measurable, not to explain it.

Worked example

Given the scenario: baseline is 80%, current is 60%, a 20 percentage-point drop, among the email-signup cohort, in the 14 days since the new flow shipped. The statement: "Onboarding completion among email-signup users has fallen from 80% to 60% in the 14 days since the new onboarding flow launched; we want to recover to at least 80% within 3 weeks, measured as the share of that cohort reaching the account-setup-complete event within 24 hours of signup." Note what's absent: no claim about WHY it dropped (a new step added friction, a bug, a segment mix shift) and no proposed fix. Those come after this statement, once investigation starts.

The same five components (target user, current state, desired state, timeframe, metric) transfer directly whether the trigger is a product metric, a data-science churn model, or a machine-learning (ML) launch: an ML repeat-purchase-drop variant of this exact scenario needs the identical five parts, just with an ML-specific metric (a model's tracked outcome) standing in for the product metric.

Trade-offs and pitfalls

The most common failure is writing a statement that is really a solution in disguise ("we need to add a progress bar to onboarding"), which locks the team onto one hypothesis before anyone has checked whether it's the right one. The second most common failure is omitting the timeframe: a problem statement without a "by when" for the desired state reads as aspirational rather than actionable, and competes poorly against work that does have a deadline. A statement that is too broad ("improve onboarding") fails the same test as no statement at all: it can't be disproven, so nobody can ever confirm it's solved.

Direct answer

"Existing business-to-business (B2B) customers on the mid-market tier, with a 6-month sales cycle, are churning at a rate that jeopardizes the annual-contract renewal base; we want to identify the dominant driver and reduce churn within two quarters." From there: stakeholders are sales (for renewal and expansion context), customer success (for account-health signal), and finance (for the retention-versus-acquisition trade-off); likely root causes to test are onboarding time-to-value, a competitor feature gap, and support responsiveness; experiments and constraints follow from which of those the data actually points to.

Structured elaboration

Stakeholders: customer success owns the account-health and usage data and the closest relationship with churning accounts; sales owns the renewal-cycle timeline and competitive intelligence from lost deals; finance owns the retention-versus-acquisition cost trade-off, since a B2B (business-to-business) product with a 6-month sales cycle makes new-customer acquisition expensive relative to retaining an existing account.

Hypothesized root causes, in the order cheapest to check: (1) slow time-to-value, meaning accounts that take longer to reach a defined "activated" milestone churn more, checkable against existing account data without new instrumentation; (2) a specific competitor feature gap, checkable against sales' loss-reason notes from recent renewal conversations; (3) support responsiveness, checkable against existing ticket-resolution-time data correlated with churned versus retained accounts.

Experiments, one per surviving hypothesis after the cheap checks above narrow the field: if time-to-value is implicated, a guided-onboarding pilot for new accounts, measuring whether faster time-to-value correlates with lower churn in the next renewal cohort; if the feature gap is implicated, that becomes a roadmap input rather than an experiment, since building a feature isn't something you A/B test before committing resources; if support responsiveness is implicated, a pilot with a dedicated response-time service-level agreement (SLA) for accounts flagged at churn risk, measuring whether faster response correlates with retention.

Constraints: a 6-month sales cycle means any fix aimed at NEW accounts won't show up in the churn numbers for two quarters at the earliest, so this problem needs both a near-term mitigation for at-risk existing accounts and a longer-term structural fix.

Trade-off: acquiring a new mid-market account costs meaningfully more, in sales cycle length alone, than retaining an existing one, which is the argument for weighting investigation and mitigation toward the churn side even if a parallel acquisition push looks more immediately visible to leadership.

Worked example

If the loss-reason review (cheap check 2) shows 60% of lost-deal notes from the past two quarters cite a specific missing integration a competitor offers, that's a stronger, more specific signal than a vague "we're losing to competition," and it turns the roadmap conversation into a concrete build-or-don't decision rather than an open-ended investigation.

A structurally similar problem, framed cross-functionally rather than as a churn diagnosis, is leading discovery for international expansion: the same discipline applies, identify the likely user segments in the new market, name the local constraints (payment methods, localization, legal and regulatory requirements), specify what data would confirm product-market fit there, and propose a phased, single-region pilot with its own measurement plan before a full-market commitment.

Trade-offs and pitfalls

The main risk in a churn problem like this is treating it as a single root cause when B2B churn is frequently multi-causal: a real fix might require addressing time-to-value AND a competitive gap simultaneously, and stopping investigation after confirming the first plausible cause risks leaving a second, equally real cause unaddressed. The six-month sales cycle constraint is easy to overlook when a fix "feels" urgent; without accounting for it, leadership may expect churn numbers to move faster than the sales-cycle math allows.