Covers end to end planning, design, and operationalization of user research studies and the concrete skills needed to collect and analyze user data. Candidates should be able to define research goals and hypotheses tied to product or business objectives; select appropriate qualitative and quantitative methods such as user interviews, contextual inquiry, diary studies, ethnographic observation, moderated and unmoderated usability testing, prototype testing, card sorting, surveys, cohort and analytics analysis, heatmap and session recording review; design screening criteria and sample size and recruitment strategies; create moderation guides, scripts, and test tasks; run studies in person and remotely; capture, transcribe, and code observations; apply analysis techniques such as thematic coding, affinity mapping, triangulation, and basic statistical checks; synthesize findings into artifacts such as personas, user journeys, jobs to be done, pain points, and prioritized recommendations; surface limitations and bias and validate findings; practice ethical research including informed consent and data privacy; and manage operational constraints such as timeline, budget, and participant access. For senior candidates include designing research strategies, defining appropriate power and sampling trade offs, creating reproducible study templates and processes, mentoring others, and describing how research choices and analysis techniques informed product or documentation decisions.
HardTechnical
44 practiced
Create a reproducible template and process that product teams can use to run a moderated usability study: include a one-page checklist, standard scripts, analysis templates (codebook and affinity map template), a timeline, and quality checks to ensure consistent moderators and reliable outputs across teams.
Sample Answer
Situation: Product teams need a consistent, reproducible way to run moderated usability studies so insights are comparable and high-quality across squads.One-page checklist (printable)- Objective & success metrics defined (task completion, SUS, time-on-task)- Target persona & recruitment criteria confirmed- Test type: moderated remote / in-person- Session length (30–60 min) & compensation- Materials ready: prototype, consent form, recording & note tools- Moderator & note-taker assigned; backup confirmed- Script loaded & pilot scheduled- Data analysis plan (codebook + affinity session date)- Storage & access for recordings/transcripts- Sign-off from PM & UX lead 48 hrs beforeStandard moderator script (abridged)- Welcome, purpose, consent: “Today we’ll ask you to…”- Warm-up questions (background, goals)- Task prompt (neutral, no leading): “Please try to…”- Probing script: “What are you thinking? Why did you do that?”- Wrap-up: SUS + open feedback- Debrief & thanks; next steps explanationAnalysis templates- Codebook (spreadsheet columns): participant ID, persona, task, observed behavior, quote, severity (1–5), usability heuristic tag, confidence- Affinity map template: Sections — Findings, Quotes (verbatim), Pain Points, Opportunity Statements, Suggested Fixes, Impact (metrics)- Output deliverables: 1-page executive summary, prioritized recommendation list (impact x effort), raw transcripts linkTimeline (example for 8 sessions)- Week 0: Define objective, recruit criteria- Week 1: Recruit participants- Week 2: Pilot session, iterate script- Week 3: Run sessions (2–3/week)- Week 4: Synthesize (codebook tagging + affinity mapping)- Week 5: Present findings & roadmap implicationsQuality checks & reproducibility- Moderator certification: 2-hour training + co-moderate 1 pilot and pass checklist- Session QA: 10% of sessions double-coded by second analyst- Inter-rater reliability target: Cohen’s kappa ≥ 0.6 for codebook tags- Standard recording & timestamping convention- Version-controlled artifacts in shared repo; changelog for scripts/codebook- Quarterly audit of 2 studies for adherence & insight qualityThis process balances speed with rigor, ensures consistent moderation, and produces analysis artifacts that engineering and stakeholders can act on.
EasyTechnical
68 practiced
You are the product manager for a mobile banking app aiming to improve onboarding conversion for new users. Define clear, measurable research goals and at least three testable hypotheses for a mixed-methods study (qualitative interviews + quantitative funnel analysis). Explain how each goal maps to a business metric and how you would validate success.
Sample Answer
Research goals (measurable)1) Identify top friction points in the first 5 minutes of onboarding — measured by % of users dropping out within first 5 minutes and qualitative frequency of reported pain points in interviews.2) Understand perceived trust/security barriers for identity verification — measured by % abandoning KYC step and interview-coded themes about trust/confusion.3) Test whether simplified flows increase completion — measured by onboarding completion rate and time-to-complete; interview sentiment on clarity.Business-metric mapping & success criteria- Drop-off in first 5 minutes → activation rate (users who complete onboarding / sign-ups). Success: reduce 5-min drop-off by ≥15% and increase activation by ≥10% in experiment cohort.- KYC abandonment → verified-account rate. Success: reduce KYC abandonment by ≥20% and increase verified-account rate by ≥12%.- Time-to-complete → time and support cost per user. Success: median time-to-complete reduced by ≥25% and support tickets about onboarding reduced by ≥30%.Three testable hypotheses (mixed methods)H1: “Reducing number of form fields on sign-up from 8 to 4 will increase completion.” Quant: A/B test measuring onboarding completion and time. Qual: interviews to confirm perceived effort lowered. Success: statistically significant lift (p<0.05) and positive interview feedback.H2: “Adding progressive disclosure for KYC (explain why each document is needed) will reduce KYC abandonment.” Quant: A/B test on KYC abandonment and verified-rate. Qual: thematic analysis showing fewer trust/confusion mentions. Success: ≥20% drop in KYC abandonment.H3: “Providing a progress bar + estimated time reduces early drop-off.” Quant: experiment measuring <5min drop-off and activation. Qual: user interviews reporting increased confidence/clarity. Success: ≥15% reduction in early drop-off.Validation approach- Quantitative: instrument funnel events (signup_start, form_submit, kyc_start, kyc_complete, onboarding_complete), run A/B tests for each hypothesis with power calc, analyze lift and p-values, segment by device/OS/cohort.- Qualitative: recruit 12–20 new users for semi-structured sessions (think-aloud during onboarding), code transcripts for friction, trust, clarity; triangulate qualitative themes with funnel anomalies.- Stakeholder reporting: deliver dashboard with cohort comparisons, exemplar quotes, prioritized backlog of fixes, and recommended rollout when metrics meet success criteria.
HardTechnical
34 practiced
You have conflicting evidence: in-depth customer interviews indicate strong demand for a new premium feature, but market research and pricing studies indicate low willingness-to-pay. Propose a hierarchical decision framework that includes exploratory experiments, MVP pricing tests, pilot releases, and clear stop/go criteria to decide whether to invest in full product development.
Sample Answer
Situation: Customer interviews (qualitative) show strong demand for a premium feature, but quantitative market/pricing studies show low willingness-to-pay. We need a staged, risk-controlled decision framework to reconcile signals before full development.Framework (hierarchical stages with objectives, methods, metrics, and stop/go criteria):1) Problem/Value Validation (2–4 weeks)- Goal: Confirm interview insights and identify specific value drivers.- Methods: 8–12 targeted interviews, jobs-to-be-done mapping, value proposition canvases, and competitive benchmarking.- Metrics: % of interviewees who rank the feature as “must-have” and specify expected outcomes; willingness to substitute current solutions.- Go: ≥50% “must-have” + consistent value drivers. Stop if <25%.2) Exploratory Monetization Experiments (4 weeks)- Goal: Test price sensitivity quickly without building product.- Methods: Van Westendorp + Gabor-Granger online surveys, pricing anchor experiments, and concept cards with purchase intent; run segmented tests (enterprise vs SMB, power users).- Metrics: Price elasticity, acceptable price range, conversion intent at target price, segment lift.- Go: Target segment(s) show acceptable price ≥ target margin and conversion intent ≥ baseline. Stop if median WTP << required price.3) Concierge / Wizard MVP (4–8 weeks)- Goal: Deliver core value manually to early users to measure real behavior and willingness to pay.- Methods: Manual or semi-automated delivery to 20–50 customers, offer time-limited paid pilot, measure usage and outcomes.- Metrics: Trial-to-paid conversion rate, retention at 30/60 days, Net Promoter Score (NPS), measured economic impact for users.- Go: Conversion ≥ 10–20% (depending on business model), retention > baseline, documented ROI cases. Stop if conversion < threshold or low retention.4) Small-Scale Paid Pilot Release (8–12 weeks)- Goal: Test productized feature with real billing and scale ~100–500 customers.- Methods: Feature flag rollout, tiered pricing, A/B pricing tests, close sales collaboration for feedback, instrumentation of revenue funnel.- Metrics: CAC payback, LTV/CAC, churn, ARPU lift, feature adoption %, qualitative feedback.- Go: LTV/CAC meets business target, lift in ARPU justifies development cost, positive qualitative signals. Stop if economics negative or adoption flat.5) Full Development Decision- Synthesize evidence: qualitative demand, revealed WTP, conversion/retention, unit economics, strategic fit.- Commit if majority of leading indicators are green and sensitivity analysis shows profitability under conservative assumptions.Governance & Risk Controls- Timebox each stage, pre-define hypotheses and thresholds, executive checkpoint after each stage.- Budget cap per stage; require updated ROI and technical estimates before next stage.- Maintain rollback plan and optional phased engineering investment tied to stage success.This framework balances qualitative enthusiasm with revealed preference and economics, de-risks investment, and yields clear stop/go decisions grounded in data.
EasyTechnical
35 practiced
Compare moderated and unmoderated usability testing. For each mode describe typical tooling, expected sample sizes, ideal use-cases (prototype fidelity, task complexity), and one situation where the other mode would be preferable.
Sample Answer
Moderated vs. Unmoderated Usability Testing — concise comparison tailored for a Product ManagerOverview:- Moderated: researcher (or facilitator) guides participants live (in-person or remote video). Good for deep qualitative insight, probing, and complex tasks.- Unmoderated: participants complete tasks independently (self-guided) using web tools. Good for scale, speed, and quantitative metrics.Tooling:- Moderated: Zoom/Teams, Lookback, UserTesting Live, Optimal Workshop for card sorts, Session recording + note-taking (Dovetail, Otter).- Unmoderated: UserTesting, PlaybookUX, Maze, UsabilityHub, Hotjar for session recordings/heatmaps, analytics for metrics.Expected sample sizes:- Moderated: 5–12 participants per user segment for qualitative discovery; iterate across rounds.- Unmoderated: 30–200+ for quantitative confidence (task completion rates, SUS scores); smaller batches (20–50) useful for quick checks.Ideal use-cases (prototype fidelity, task complexity):- Moderated: high-complexity tasks, early concept validation, high-fidelity prototypes, exploratory interviews, when you need to probe "why" and follow-up questions.- Unmoderated: lower-to-medium complexity tasks, mid/low-fidelity prototypes or live product A/B tests, rapid benchmarking, large-sample metrics.When the other mode would be preferable:- Use moderated instead of unmoderated when tasks are complex, ambiguous, or require probing (e.g., onboarding flows with conditional logic) because automated prompts can miss nuance.- Use unmoderated instead of moderated when you need scale, speed, or realistic in-context behavior (e.g., measuring task completion on production checkout across many demographics) and budget/time are constrained.Practical tip: combine both—start moderated for discovery and iterate with unmoderated for validation and metrics.
EasyTechnical
40 practiced
Describe the process of synthesizing qualitative findings into personas, user journeys, and 'jobs to be done' artifacts. Include which data sources you'd use, what fields you'd include in a persona, and how to avoid stereotyping or overgeneralization.
Sample Answer
Start with goals and inputs: clarify what decisions the artifacts should inform (roadmap, UX flows, prioritization). Collect and triangulate data: primary qualitative (user interviews, contextual inquiry, diary studies, usability tests), quantitative complements (analytics, support tickets, NPS, feature usage), and market sources (surveys, competitor reviews).Synthesis process:- Affinity mapping: cluster quotes, behaviors, pain points and needs.- Identify patterns: group by goals, motivations, context, and constraints.- Create artifacts: personas, user journeys, and JTBD statements from those clusters. - Persona: name, photo (archetypal), role & demographics (only if relevant), goals & motivations, top tasks, pain points, tech comfort, preferred channels, context of use, success metrics, key quotes, behavioral patterns, priority level. - User journey: stages (discover → evaluate → use → support), user goals per stage, actions, touchpoints, emotions, pain points, opportunities and metrics. - JTBD: concise jobs framed as “When [situation], I want to [motivation], so I can [outcome]” plus acceptance criteria and barriers.Avoid stereotyping/overgeneralization:- Base artifacts on aggregated evidence (quotes counts, cross-source validation), not single anecdotes.- Show variability: include edge cases, confidence level, and segments where behaviors diverge.- Use data tags: link persona traits to source excerpts and frequency.- Avoid demographic assumptions as proxies for behavior; focus on goals and context.- Validate: run quick concept tests with users and stakeholders; iterate.Outcome: artifacts that are evidence-backed, actionable for roadmap decisions, and maintainable as new data arrives.
Unlock Full Question Bank
Get access to hundreds of User Research Methods and Execution interview questions and detailed answers.