Consultative Discovery and Requirements Gathering Questions

Direct answer

I keep a written assumptions register that the stakeholders can see, give every assumption an owner, a way to test it and a review date, and label everything I deliver with the assumptions it rests on. An assumption becomes a requirement only through explicit confirmation by a named person, never through silence or because I built on it.

The register (one row per assumption)

Worked example: weekly revenue ETL (extract, transform, load: moving and reshaping data on a schedule)

Ordering rule: attack low confidence and high impact first. Marketing spend data missing? I state an assumption (A5 above: spend is spread evenly over days until invoices arrive), label the output "estimate", and plan to back-fill actuals (replace the estimate with the real figures once invoices arrive) and publish a difference report (a short table of estimate versus actual for each week, so readers see how far off the estimate was).

How I stop assumptions turning into requirements

• Every deliverable carries the line "based on assumptions A1 to A5, unconfirmed".
• Confirmation is recorded with a name and a date in the register.
• Each row has an expiry date: unconfirmed by then means it is escalated, not forgotten.
• In code or config I tag the assumption ID, so it can be found when it is replaced. For example, a comment above the SQL: -- A1 (unconfirmed): revenue = gross amount minus refunds.
• I use the register in negotiation: "Delivery by Friday is possible if A1 and A3 are confirmed today. Otherwise I deliver on stated assumptions and the risk is yours to accept in writing."

Techniques to surface hidden assumptions

Turning interviews into a ranked list: collect every assumption heard, rate confidence and impact, and test the low-confidence, high-impact ones first.

Direct answer

A strong answer names a specific project, shows the ambiguity was a concrete phrase (not "unclear requirements" in general), says exactly how and when the team found out, describes a re-scope that involved the stakeholders rather than blame, and ends with habits you now follow that you can show.

How to structure the story (STAR: situation, task, action, result)

• Situation and task: one or two sentences on the goal and your role.
• The ambiguity: quote the vague phrase the team built from.
• The moment you found out: when, how, and what finally exposed it.
• Action: who you brought together, what you stopped, what you kept.
• Result: honest, mostly qualitative. Own the cost.
• Change: specific practices, not virtues.

Worked example (an illustrative story shape; use your own project)

• Situation: A team was building a feature for an operations lead who asked for "a way to get order data out of the system". The team had 8 weeks planned.
• Ambiguity: "get data out" was read as a scheduled spreadsheet export. Nobody asked what she did with the file.
• The moment we found out: At the demo in week 5, I asked her to open the export in her real workflow. She needed to reconcile orders with the finance system every hour, so a full file each night was useless: she needed only the changes since the last run, delivered automatically.
• Action: I stopped adding export options. Within two days I ran a 45-minute session with her and the finance owner, using her last real reconciliation as the script. We agreed what to keep (the file export, which the audit team did need), what to cut (scheduled email variants), and what to add (an incremental feed). I wrote the new scope as measurable sentences, such as "changes since the last pull are available within 15 minutes", and had both owners confirm them in writing.
• Result: The feed shipped later than the original plan (the delay was real, and I told the sponsor the new date the same day). The manual reconciliation stopped. Three weeks of work on export options were largely wasted.
• What I do differently: (1) Ask "show me the last time you did this by hand" before writing anything. (2) Write each requirement as who, when and how measured, and read it back in writing. (3) Put real data in front of the requester in the first two weeks, not week 5. (4) Keep a list of assumptions and review it at every checkpoint.

Pitfalls

• Blaming the stakeholder ("they did not know what they wanted"). Interviewers read that as no ownership.
• A story with no moment of finding out, so it sounds like it was always going to be fine.
• Invented precision (exact percentages). Use only numbers you can defend.
• Ending with "communicate more" instead of practices someone could watch you do.

Direct answer

An "active user" is a user who performed a defined qualifying action within a defined window. Product and growth disagree because each is implicitly defining a different action, window or population. I would get agreement by writing the definition as four explicit parts, showing both camps their own numbers side by side, and agreeing on which decision the definition serves. Then I test whether the choice matters by computing both definitions on the same data and checking whether they lead to different decisions.

Getting to one definition (a 30-minute alignment meeting)

• Minutes 0-5: state the decision the number feeds (for example "is the new onboarding working?"). Different decisions can legitimately need different metrics.
• Minutes 5-15: fill in the four parts on a shared page: qualifying event (login, or a core action such as creating a project), window (daily, weekly, 28 days), population (all accounts, or excluding staff and test accounts), and time rule (which timezone, which day the week starts: a "time rule").
• Minutes 15-25: show both candidate definitions run on last month's data.
• Minutes 25-30: pick one as "the" definition for the dashboard, name an owner, and agree that the other keeps a different name (for example "logged-in users").

Does the choice matter? (illustrative numbers)

Definition A (product): any login in the week. Definition B (growth): at least one core action in the week.

4,200 / 4,000 = 1.05 and 1,350 / 1,500 = 0.90. The two definitions point in opposite directions, so the choice changes the story: logins are up while real usage is down. That is a decision-relevant difference and worth the debate. If both had moved within a point or two of each other, I would pick the cheaper one and move on.

Beyond the headline count I also compare the overlap (here, the users counted by both definitions, and those counted by A but not B) and whether the gap is concentrated in one segment.

The same discipline for neighbouring definitions

• Conversion: state the numerator (who counts as converted), the denominator (who was eligible, and when they entered), and how returns or refunds are treated. Example (illustrative): of 1,000 users who started a trial in March, 200 paid, so 200 / 1,000 = 20%. Using all 1,600 users who ever signed up as the denominator gives 200 / 1,600 = 12.5%. Same sales, two answers.
• Weekly-active with missing timestamps: measure the share of events with no timestamp first. Then either exclude them and report the exclusion, or fall back to ingestion time (when the event reached our database, rather than when the user did it), and check both versions to see if the answer changes. Example (illustrative): if 10,000 of 100,000 weekly events (10%) lack a timestamp, I compute the metric both ways and see whether the weekly count moves by more than a point or two.
• Lifetime value (LTV, the total revenue expected from a customer): list inputs (revenue, margin, horizon, discount) so two analysts do not use different ones. Margin is the share of revenue left after costs, horizon is how many months you count, and a discount reduces the weight of later months because future money is worth less. Example (illustrative, no discount): $30 a month x 70% margin x 12 months = $252; with a 24-month horizon the same customer is worth $504.
• Cohort boundaries: a cohort is a group of users who start in the same period. Decide which date puts a user in one (signup, first purchase), in which timezone. Example: a signup at 23:30 UTC on Sunday is already Monday in Sydney, so, with weeks starting on Monday, the two timezones put that user in different weekly cohorts (with Sunday-start weeks the same signup would land in the same week, which is why the week-start rule belongs in the definition).
• Customer-segment labels (a segment is a group of customers sharing a trait, such as plan or size): write the rule that builds each label, date-stamp it, and version it so a label means one thing over time.

Trade-offs and pitfalls

• Declaring a winner by seniority rather than by the decision served.
• Keeping two metrics under one name, which guarantees the argument returns.
• What would change my call: if the definitions never diverge on past data, I choose the simpler one and document the other.

Direct answer

"Improve retention" is a goal, not a request. In the first ten minutes I want to learn what decision this feeds, retention of which users, what counts as "retained", and what the number is today versus the target. I push back or reframe when there is no decision attached, no agreed definition or baseline, or when the data cannot answer the question being asked.

Retention is the share of users who keep coming back or keep using the product over a period. A cohort is a group of users who share a start point, such as signing up in the same week.

The questions, in order

1. Trigger and decision: "What prompted this? Did a number move, or is there a target from above? What will you decide based on my work?"
2. Whose retention: "New signups, paying customers, one plan or channel?"
3. Definition: "What counts as retained: opening the app, or doing the key action? Over what window, week four, day 30, a year?"
4. Baseline and target: "What is the number today, who measured it and how, and what number by when counts as success?"
5. Past thinking: "What do you already suspect, and what has been tried?"
6. Deliverable and deadline: "A diagnosis, a forecast, an experiment plan or a dashboard? For whom, by when?"

When I push back or reframe

• No decision attached: I ask for the decision first. Analysis without one gets filed.
• No definition or baseline: nobody could tell whether the number moved, so I propose a definition in writing.
• "Improve" is not a question the data answers: I reframe it, for example "which signup channel's week-four retention fell since last quarter, and which early behaviours separate returners from leavers?"
• A cause is expected from observational data: data we simply collected (nobody was randomly assigned) shows association, not cause. To establish cause we would need an A/B test (a controlled experiment where users are randomly given different versions). I say so up front.
• The deadline does not fit the question: I offer a smaller version and list its caveats.

Worked example (illustrative numbers)

Take a cohort of 1,000 signups from one week. If "retained" means any login in week four and 400 did, retention is 400 / 1,000 = 40%. If it means creating at least one report in week four and 250 did, it is 250 / 1,000 = 25%. Same users, two answers, so the definition has to be fixed before any work starts.

The reframed brief I would send back: "Find which signup channel's week-four retention (created a report) fell versus last quarter and recommend one change to test. A short written summary by Friday, with caveats stated."

Trade-offs and pitfalls

• Asking six questions in a row can feel like resistance. Pair each with why it helps: "so I answer the question you actually have".
• Accepting a vague brief to look responsive creates rework weeks later.
• Over-scoping the first reply (a full model) when a fast descriptive cut (a plain summary of what already happened, such as counts by channel, with no model or experiment) would answer the decision.

Direct answer

I take notes that separate what people said from what I saw and what I think it means, and I keep a running list of decisions, assumptions, open questions and action items with owners. I send a clean version the same day (at the latest within 24 hours) and ask for corrections by a named date. Notes that sit in a personal notebook cannot be checked by anyone else.

What I capture, beyond the words

Alongside that: decisions made, assumptions I am relying on, open questions, and actions with owner and date. Keep verbatim quotes and inferred needs apart, so a reader can tell the stakeholder's words from my guess.

A findings record (CRM-style, one row per finding)

Customer relationship management (CRM) tools store a structured record for each contact, account or deal, with every interaction logged against it. The same idea works here, with one row per finding: ID, date, source (person and role), the finding, its type (need, constraint or assumption), the supporting quote, confidence, status and owner. Searching "all constraints from finance" is then easy.

For business intelligence (BI) requirements meetings

A fixed template helps: the report's purpose and audience, the metric definitions, filters, refresh frequency, source tables, and who signs off. Add reproducibility notes: which dashboard or data snapshot (with its date) we looked at, the query or its version, and the assumptions behind any number quoted in the room. Someone should be able to rerun what was shown.

Worked example

Said: "Active customer means anyone who ordered recently." (Priya, finance). Observed: she opened a spreadsheet that excludes refunds. Inferred (mine): "recently" may be 90 days, to confirm. Open question: is 90 days the agreed window? Action: I confirm the window with Priya by Friday.

How soon to share

The same day, while people still remember the conversation. Include the line "Reply by Friday if anything is wrong; otherwise I will treat this as agreed." That line covers the recorded decisions and quotes. Items marked Inferred or Open question stay open until the named person confirms them, so silence never turns a guess into a requirement.

Trade-offs and pitfalls

• Transcribing everything instead of marking what matters.
• Mixing quotes and inference, so guesses harden into "requirements".
• Actions without owners or dates.
• Writing in private shorthand nobody else can read.