User Research and Discovery Questions
Planning and running user research and discovery to inform product direction. Covers research methodology, prioritizing a research roadmap, integrating findings into strategy, and partnering research with design and product. Assesses rigor in generating and applying user insight.
Explain when to use low-fidelity (paper), mid-fidelity (clickable) and high-fidelity prototypes in discovery. For each fidelity level, describe the minimum effort, typical validation questions it's good for, risks, and one example scenario where it is the right choice.
Sample Answer
Low-fidelity (paper/sketch)
- Minimum effort: 10–60 minutes; hand-drawn screens, flows, paper cutouts. Cheap to iterate during interviews or workshops.
- Good validation questions: Do users understand the concept/flow? Is the problem/opportunity framed correctly? Which features matter most?
- Risks: Users may dismiss it as “not real”; can’t test interaction timing or microcopy; ambiguous visuals can lead to misleading feedback.
- Example: Early discovery for a new onboarding flow. Test whether users grasp steps and value proposition before building clickables.
Mid-fidelity (clickable prototypes)
- Minimum effort: Hours–days; digital mockups with basic styling and clickable hotspots (Figma/Proto.io).
- Good validation questions: Is navigation intuitive? Does the task flow accomplish goals? Where do users hesitate/confuse?
- Risks: Looks more finished, which can bias feedback toward polish; limited backend realism (data/state edge cases).
- Example: Validating an account settings workflow with target users to refine labels, error states, and branching paths before engineering effort.
High-fidelity (pixel-perfect, interactive with realistic data)
- Minimum effort: Days–weeks; detailed UI, polished copy, realistic interactions, maybe instrumented for analytics.
- Good validation questions: Will users convert/complete tasks at acceptable rates? Do microinteractions, copy, and performance affect outcomes? Usability at scale.
- Risks: High cost and sunk effort if concept changes; stakeholders may conflate prototype with final product; longer iteration cycles.
- Example: Pre-launch A/B test of a redesigned checkout with realistic payment flows to estimate conversion lift and technical edge cases.
Use fidelity progressively: start low to validate problem and concepts, move to mid to refine flows, and use high to de-risk launch and measure real outcomes.
Your team shipped a redesigned checkout flow based on a research insight, and now needs to know whether it actually worked. Walk through how you'd turn that insight into a measurable KPI, design the experiment or metric-tracking plan to test it, and explain how you'd set a baseline, target, and timescale, plus what qualitative follow-up you'd run alongside the quantitative read to catch anything the metric alone would miss.
Sample Answer
Situation & chosen insight
I prioritized the insight: "High first-time onboarding drop-off on step 2 (profile details) is blocking activation." I translate that into a measurable KPI and an experiment.
KPI & baseline
- Primary KPI: Onboarding completion rate (users who finish all onboarding steps within 7 days of signup).
- Baseline: 42% completion (last 30 days).
Target & timescale
- Target: Increase to 55% within 8 weeks post-launch (roughly +13 percentage points, "pp" for short).
- Secondary metrics: Time to complete onboarding, conversion to activated user (first paid action), and drop-off rate at step 2.
Experiment design
- Variant A = current flow (control). Variant B = redesigned flow (progressive disclosure + inline validation + save-for-later).
- Randomized A/B test with a 50/50 UA split (User Allocation split: the percentage of users randomly routed into each variant).
- Minimum sample size: sized to reliably detect a conservative minimum lift of 4 percentage points at 80% statistical power. Power is the probability the test actually detects a real effect if one truly exists; 80% is the standard bar, meaning we accept a 20% chance of missing a real lift in exchange for a smaller, cheaper sample than 90% or 95% power would need.
- The arithmetic: using the standard two-proportion sample-size formula, n = 2 x (z_alpha/2 + z_beta)^2 x p_bar(1-p_bar) / (p1-p2)^2, with baseline p1 = 0.42, target p2 = 0.46 (a 4pp lift), p_bar = 0.44, z_alpha/2 = 1.96 (95% two-sided confidence) and z_beta = 0.84 (80% power): n = 2 x (2.80)^2 x (0.44 x 0.56) / (0.04)^2 = 2 x 7.84 x 0.2464 / 0.0016, about 2,415, which rounds to the N ~ 2,400 per arm target (adjust with real traffic and recheck as data comes in).
- Run for at least 2 full weeks and until sample size/statistical significance reached; monitor for seasonality.
Measurement & tracking plan
- Instrument events: signup_started, onboarding_step_completed (with step id), onboarding_completed, time stamps, user cohort id.
- Build dashboard showing funnel, completion rate, step-level drop-offs, time-to-complete, and segmented by device, channel, and new vs returning.
- Statistical test: two-proportion z-test with confidence intervals; also track lift and p-value (the probability of seeing a difference this large by chance alone if the redesign actually had no effect; conventionally, below 0.05 is treated as a significant result).
Qualitative follow-ups
- Recruit users from both variants who dropped off and who completed for short usability interviews (5–8 users per cohort).
- Review session recordings and heatmaps focused on step 2.
- Use feedback to iterate on microcopy, input affordances, and error messaging.
Outcome interpretation
- If KPI target met with significance and secondary metrics improve, roll out change. If lift is small or mixed, use qualitative insights to refine and run a follow-up test.
You're kicking off research for a new feature with a roadmap review in one week. Walk through how you'd decide whether to run qualitative interviews, send a quantitative survey, or do both, and explain what would change your answer if you only had two days instead of a week.
Sample Answer
Direct answer
With a week, the deciding factor is what kind of unknown you have: if you don't know why users behave a certain way or what shape the solution should take, run qualitative interviews first, since a genuinely unknown problem needs exploration before it can be measured. If you already understand the problem and need to know how many people are affected or which option performs better, a quantitative survey is the stronger first move. With a full week, a short round of each, a handful of interviews to sharpen the survey questions, then the survey to size what you found, usually beats either method alone. With only two days, drop the survey: getting a valid, representative response volume takes longer than two days to field, so a rushed survey produces a number that looks precise but isn't trustworthy. Run 4 to 5 rapid interviews instead, and pull whatever analytics or support-ticket data already exists.
Structured elaboration
Three questions to ask before choosing a method:
- What is actually unknown? An exploratory unknown ("why" or "what") points to qualitative work; a confirmatory unknown ("how many" or "which option wins") points to quantitative work.
- How much time do you actually have to reach a valid sample? Interviews can be scheduled and run within days; a survey needs days just to field and collect a stable response rate before you can trust the resulting percentage.
- Does the answer already exist? Check product analytics, support tickets, and past research before commissioning anything new. Often the quantitative half of the answer is already sitting in existing data, and the real gap is a handful of interviews to explain it.
When each method fits:
- Qualitative fits questions like "why are new users confused during onboarding," a brand-new feature concept with no existing usage data, or understanding a workflow nobody has documented yet.
- Quantitative fits questions like "what percentage of users would use this" or "which of two designs converts better," and sizing a problem you already understand well enough to measure before committing engineering time.
- Both together fit high-stakes decisions where you need to explain why something is happening and defend a number to leadership.
Worked example
One-week plan: day 1, recruit and draft a 5-question interview guide; days 2 to 3, run 6 contextual interviews (about 45 minutes each) about the workflow the feature targets; day 3 evening, draft a 6-question survey informed by the interview themes; days 4 to 5, field the survey to existing users through an in-app prompt, targeting at least 150 responses for a stable percentage; day 6, analyze; day 7, present both the synthesized themes and the survey top-line numbers to the roadmap review.
Two-day plan: day 1 morning, pull existing analytics and support-ticket data on the problem (a free, instant signal); day 1 afternoon and evening, run 4 to 5 rapid interviews with existing customers reached through customer success or a community channel, with no formal recruiting process; day 2, synthesize and present the roadmap review with the directional qualitative signal plus the existing analytics, explicitly labeled as directional rather than statistically powered. Skip a freshly launched survey entirely: one fielded for a single day only reaches the fastest-responding, most-online slice of users, which biases the result in a way that's easy to mistake for a real number.
Trade-offs and pitfalls
The most common mistake is treating a rushed two-day survey as if it carried the same evidentiary weight as a properly fielded one. Qualitative-only research skips confirming how common a pain point actually is, risking a build for a vocal minority; quantitative-only research skips the "why," risking solving the wrong problem precisely and confidently. And the biggest pitfall of all is picking the method you personally prefer running rather than the one the type of unknown actually calls for.
You've uncovered a persistent unmet need in a minority user segment, but serving it would cannibalize a profitable core flow and require heavy engineering investment. A vocal, loyal group keeps asking for it even though the addressable market looks small. Walk through how you'd investigate the business and user trade-offs, build a data-informed business case, and recommend a sequencing plan to minimize risk while testing whether the opportunity is real.
Sample Answer
In short
Treat this as two separate questions that get conflated under pressure: is the underlying need real and sizeable enough to matter, and separately, can it be tested without committing to the full, expensive build. Answer the first with a data-informed business case, and the second with a staged, reversible pilot, so the vocal minority's persistence doesn't substitute for either.
Framework
Investigate the trade-off:
- Size the segment: how many users are affected, how often they hit the need, and their current lifetime value, or LTV (the total revenue a typical user generates over their relationship with the product), versus their value if the need were met.
- Map the cannibalization risk concretely: estimate what fraction of core-flow usage or revenue would plausibly shift to the new option, not just "some users might switch," and compare that against the net new value the change could unlock.
- Validate depth of need with qualitative research: interviews and, if feasible, a lightweight usability test of a rough concept, specifically probing whether this is a "must have, blocking" need or a "nice to have, workaround exists" need. A small group with a genuinely blocking need can be worth more strategically than segment size alone suggests.
Build the business case:
- State the hypothesis explicitly and testably, for example "a dedicated option for this segment converts a meaningful share of them, and total cannibalization of the core flow stays below a threshold we can absorb."
- Build a simple model with a best, likely, and worst case: incremental revenue or retention from serving the segment, minus the revenue lost to cannibalization, minus build and support cost, over a defined time horizon.
- Decide the metric that would prove or disprove the hypothesis before running anything, net revenue impact, not just adoption by the vocal segment.
Sequencing to minimize risk:
- Start with the cheapest possible test of demand, a fake-door or waitlist test, to gauge real interest before building anything.
- If demand is real, pilot behind a feature flag to a small percentage, 5 to 10%, of the eligible segment, instrumented to measure both adoption of the new option and any change in core-flow usage among that same cohort.
- Expand gradually, by segment or geography, only while the net metric, new value minus cannibalized value, stays positive, with an explicit rollback plan if it doesn't.
Worked example
Suppose the segment is 4% of the user base, and internal estimates suggest 25% of that segment's core-flow usage could shift to the new option if built, a plausible, not certain, cannibalization rate. A pilot to 8% of that segment over 6 weeks shows 60% adoption of the new option among pilot participants, with only 10%, well under the estimated 25%, actually reducing their core-flow usage, because most were using the new option for a genuinely different job rather than replacing the old one. That result argues the cannibalization fear was overstated for this segment and supports a wider rollout with continued monitoring.
Trade-offs and pitfalls
The most common mistake is letting the vocal group's persistence stand in for evidence of size or of the true cannibalization rate; a loud minority and a large, at-risk minority are not the same thing, and only sizing and a real pilot can tell them apart. The second is treating the initial cannibalization estimate as fixed instead of something the pilot exists to test: the whole point of instrumenting the pilot cohort's core-flow usage is to replace the estimate with a measured number before scaling.
As a senior researcher, you're given competitor analysis, recurring support-ticket themes, a declining NPS trend, and a recent round of user interviews. Synthesize these into three strategic product initiatives, rank them by impact and feasibility, and give one success metric you'd track for each.
Sample Answer
In short
The job is to find where the four signal sources agree, not to average them: a theme that shows up in support tickets, the NPS (Net Promoter Score, a survey-based loyalty metric) decline, competitor gaps, and interview quotes at the same time is far more trustworthy than any one source alone, and that convergence should drive the ranking, not which signal is loudest.
Framework
Cross-reference the four sources for overlap before naming initiatives: cluster support-ticket themes, map interview quotes to the same clusters, check whether the NPS decline concentrates in a particular segment or moment (new users versus power users, a specific workflow), and note where a competitor already solves the same pain. A theme appearing in 3 or 4 of the sources is a strong candidate; a theme from only one source needs more evidence before it becomes an initiative.
Three ranked initiatives, ranked by impact times feasibility:
- Fix the highest-friction part of onboarding (impact 9, feasibility 8, score 9 x 8 = 72): if ticket volume and interview quotes both concentrate on the first-week experience and NPS is weakest among new users, this is usually the highest-confidence, fastest-to-ship initiative. Success metric: 7-day activation rate, the percent of new users completing the core first action.
- Resolve the top reliability or performance complaint (impact 8, feasibility 6, score 8 x 6 = 48): if tickets and interviews both point to a specific failure mode, a sync error, a slow step, that frustrates existing, higher-value users, this addresses churn risk directly. Success metric: related support tickets per 1,000 monthly active users, or MAU (the count of distinct users active in a typical month).
- Close the clearest competitive gap for power users (impact 6, feasibility 6, score 6 x 6 = 36): if interviews and competitor analysis both name a specific missing capability that power users want, this is a retention and win-back play rather than a growth play. Success metric: adoption of the new capability among the top decile of most active users within 30 days.
Worked example
Suppose support tickets show 22% of volume relates to account-setup confusion, interviews (8 of 12 participants) independently raise the same confusion unprompted, and NPS is 15 points lower among users under 30 days old than among tenured users. All three signals converge on onboarding, so it ranks first with high confidence. By contrast, a "missing export feature" appears in only 2 of the 12 interviews and isn't visible in ticket data at all; it goes on a watch list rather than becoming one of the three ranked initiatives, since a single source isn't enough to commit resources against.
Trade-offs and pitfalls
The main trap is anchoring on whichever source is most vivid, usually a strongly worded interview quote, and retrofitting the other data to support it; check convergence across sources deliberately rather than trusting your first impression. The second trap is presenting these as three independent tactical fixes instead of naming what they imply about the competitive landscape and where the product should place longer-term bets, a more strategic framing worth using when the audience is executives rather than the immediate product team.
Unlock Full Question Bank
Get access to all 7 User Research and Discovery interview questions and detailed answers.
Sign in to ContinueJoin thousands of developers preparing for their dream job.