User Research and Problem Framing Questions

Covers the end to end practice of uncovering, defining, and validating the true problem before designing solutions. Includes techniques for framing ambiguous challenges, performing root cause analysis, and translating business needs into clear problem statements and research objectives. Covers designing and prioritizing research activities including stakeholder and contextual interviews, user interviews, surveys, field research, observational studies, analytics review, competitive and market analysis, and selecting appropriate qualitative and quantitative methods and sample considerations. Emphasizes hypothesis driven research, rapid prototypes and experiments, ethical practice, and using analytics to validate insights. Describes how to set clear success criteria and key performance indicators, surface stakeholder assumptions and constraints, convert vague needs into testable research questions and hypotheses, and produce deliverables such as research plans, personas, user journeys, empathy maps, prioritized findings, and actionable recommendations that inform decisions and design goals.

EasyTechnical

39 practiced

You plan 12 usability test sessions for a consumer photo-sharing app. Explain a sampling approach that balances diversity and relevance: list inclusion/exclusion criteria, suggested quotas (e.g., by experience level, device), and two recruitment channels you would use to reach participants.

MediumTechnical

34 practiced

Given 2 weeks, a $5k budget, and the following candidate activities: stakeholder interviews, analytics audit, six user interviews, and a five-participant moderated usability test, prioritize these activities into an execution plan. Justify the order, expected outcomes of each activity, and outline a contingency plan if recruitment is delayed.

Sample Answer

**Plan (2 weeks, $5K)**

Day 1–2 — Stakeholder interviews (2–3 sessions)- Purpose: Align on business goals, success metrics, constraints, and hypotheses.- Expected outcome: Prioritized research questions, acceptance criteria, list of technical/legal constraints to shape recruitment and tasks.- Cost: internal time (minimal cash)

Day 3 — Analytics audit (1 day)- Purpose: Quickly surface high-frequency flows, drop-off points, and top pages/features to target research.- Expected outcome: Data-driven target tasks for usability test and recruitment screener; quick win metrics (e.g., pages with highest abandonment).- Cost: internal/analyst time

Day 4–8 — User interviews (6 interviews, remote, 45–60m)- Purpose: Qualitative discovery to uncover mental models, goals, pain points.- Expected outcome: Themes, quotes, user types to inform prototype and task scenarios; 6 interview slots at ~$75–150 incentive = $450–900- Cost: ~$900

Day 9–13 — Prototype + usability test prep- Build low-fi or clickable prototype focused on highest-impact flows from analytics + interviews; create test script and recruitment screener.- Cost: design time, prototyping tools (minimal cash)

Day 14 — Moderated usability test (5 participants)- Purpose: Validate flows, observe usability issues, task success, and time on task.- Expected outcome: Top usability issues prioritized by severity and impact; recommended fixes.- Cost: participant incentives $100 x5 = $500; remote testing tools $200; remaining budget for recruitment agency if needed.

Total estimated spend: ~$2k–3k (leaving buffer for last-minute recruitment or recordings)

**Justification**- Stakeholders first ensure research is actionable and scoped correctly.- Analytics prior to heavy recruitment makes tests targeted and efficient.- Interviews inform design decisions for a focused prototype.- Testing last validates specific changes and yields concrete recommendations.

**Contingency: Recruitment delays**- Pivot to guerrilla testing with internal employees or existing customers identified via analytics (quick vetting).- Use remote unmoderated tests (e.g., Maze, UserTesting) for rapid signals with recruited panels.- If participants drop, prioritize the most critical tasks and run fewer moderated sessions combined with additional short user interviews to broaden qualitative evidence.- Communicate timeline impacts and trade-offs to stakeholders and request small budget reallocation to a recruiting agency if needed.

HardTechnical

34 practiced

Analytics show Feature X increases weekly active users by 12%, yet qualitative interviews report consistent confusion and negative sentiment about the feature. Provide a step-by-step plan to reconcile these conflicting signals: what segmentation and cohort analyses you'd run, targeted qualitative follow-ups, experiments or variant tests you'd design, and the decision criteria (iterate, rollback, or evolve) you would use.

Sample Answer

**Situation framing (1–2 lines)**Feature X drives +12% WAU overall but interviews show confusion/negative sentiment. Goal: determine who benefits, why others are confused, and decide iterate/rollback/evolve.

**1) Segmentation & cohort analyses**- Run WAU lift by segment: new vs returning users, power users, novice users, geography, platform (iOS/Android/web), acquisition channel.- Cohort retention and engagement: 1-, 7-, 28-day retention and feature usage frequency for users exposed vs unexposed.- Behavioral funnels: entry point → Feature X activation → core task completion → downstream metric (purchase/share).- Time-based decay: does lift persist or is it short-term curiosity?

**2) Targeted qualitative follow-ups**- Recruit users from segments with high lift + negative sentiment and low lift + negative sentiment.- Tasks: moderated usability test with think-aloud on primary flows; measure task success, time-on-task, and emotional valence.- Contextual interviews: observe in-home/work environment for real-world use; probe mental model mismatches.- Collect verbatim quotes and map pain points to UI elements and copy.

**3) Experiments / variant tests**- A/B variants: (A) current X, (B) streamlined UI with clearer affordances, (C) explanatory onboarding/tooltips, (D) reduced or delayed placement.- Add micro-experiments: hover/tool-tip vs modal onboarding; different copyframes; progressive disclosure for novice users.- Measure both quantitative (WAU, retention, task success) and qualitative (post-task SUS, sentiment scores).

**4) Decision criteria**- Evolve: keep if lift persists in target segments AND usability tests show fixable pain points (low severity) and experiments improve sentiment without losing lift.- Iterate: if lift concentrated but sentiment salvageable via UX fixes (onboarding, copy, IA) and A/B shows maintained lift + improved satisfaction.- Rollback: if lift is ephemeral, causes high-severity usability failures, or harms core task completion in key cohorts.- Concrete thresholds: maintain if retention delta ≥ +8% at 28 days for target cohort and SUS increases ≥ 10 points post-fix; rollback if core-task success drops > 10% or NPS drops significantly.

**5) Communication & next steps**- Share cross-functional brief with analytics, PM, and engineering; propose prioritized UX fixes; run 2–4 week experiments; reassess with mixed-methods results and decide based on thresholds.

MediumTechnical

37 practiced

You observe large drop-off in onboarding at the 'add payment method' step. Describe five analytic explorations you would run (be specific: segments, event sequences, funnel windows), the metrics you would compute for each, and how you would pair each analytics check with a qualitative follow-up to validate explanations.

Sample Answer

**Overview**

I’d run five targeted quantitative explorations, each paired with a qualitative check to validate UX hypotheses.

1) **Overall funnel & drop magnitude**- Segments: all users, new vs returning, web vs mobile- Sequence: Start onboarding → Add payment screen → Success- Window: same-session; 24–72 hour re-attempts- Metrics: conversion rate to add-payment, drop-off %, time-on-step median- Qualitative follow-up: moderated usability sessions where participants complete onboarding while thinking aloud to surface friction.

2) **Error & validation failure analysis**- Segments: device, browser, locale, card type- Sequence: payment form field events → form-submit → error events- Window: same-session, real-time event stream- Metrics: error rate per field, most common error messages, submit retry count- Qualitative follow-up: intercept surveys after failures asking what went wrong; lab test with error scenarios.

3) **Time & hesitation patterns**- Segments: first-time vs returning, assistive tech users- Sequence: enter payment screen → field focus/blur → submit- Window: session + 5 minutes- Metrics: time-to-first-input, idle periods >15s, keystroke/backspace rates- Qualitative follow-up: remote unmoderated session recordings and follow-up interviews to probe confusion.

4) **Drop by acquisition source & intent**- Segments: marketing source, paid vs organic, referral vs email- Sequence: landing → onboarding → add payment- Window: 7-day funnel- Metrics: step conversion by source, CPC vs conversion quality- Qualitative follow-up: targeted user interviews for specific cohorts (e.g., paid ad converts) to understand expectations mismatch.

5) **Feature-flag / A/B / UI variation check**- Segments: variant groups, app version- Sequence: add-payment UI variant → submit- Window: experiment duration- Metrics: lift/loss in conversion, interaction patterns per variant- Qualitative follow-up: usability A/B follow-up interviews and preference tests to explain behavioral differences.

For all checks also verify event instrumentation and sampling bias before trusting results.

HardTechnical

31 practiced

You plan an A/B test to increase onboarding completion from 60% to 66% (absolute improvement). For 80% statistical power and alpha = 0.05, estimate the required sample size per variant and explain the assumptions and steps of your calculation (including which formula or approximations you use). Also explain how you would handle multiple metrics, sequential looks, and early stopping rules.

Sample Answer

**Answer (UX Designer perspective)**

**Goal & summary**We want to detect an increase from p1 = 0.60 to p2 = 0.66 (absolute +6 pp) with 80% power and alpha = 0.05. Using the two-proportion z-test normal approximation, the required sample is about 1,014 users per variant.

**Calculation steps & formula**Assumptions: independent randomized assignment, binary outcome (completed onboarding), stable baseline, no heavy clustering.

Key formula (per-group sample size for two-sided test):

text

n = [ ( z_{1-α/2} * sqrt(2 * p̄ * (1-p̄)) + z_{power} * sqrt(p1*(1-p1) + p2*(1-p2)) )^2 ] / (p2 - p1)^2

Where p̄ = (p1 + p2) / 2.

Plugging numbers:

text

p1 = 0.60
p2 = 0.66
p̄ = 0.63
z_{1-α/2} = 1.96
z_{power} = 0.84

Compute intermediate terms: sqrt(2*p̄*(1-p̄)) ≈ 0.683; sqrt(p1*(1-p1)+p2*(1-p2)) ≈ 0.682. Numerator ≈ (1.96*0.683 + 0.84*0.682)^2 ≈ 3.649. Denominator = 0.06^2 = 0.0036. So n ≈ 3.649 / 0.0036 ≈ 1,014 per group.

Practical note: add inflation for expected attrition, vs. multiple segments, or clustering (e.g., if users can have multiple sessions).

**Handling multiple metrics**- Pre-specify one primary metric (onboarding completion). Treat others as secondary/exploratory.- If claiming statistical significance across multiple primary metrics, control family-wise error: use Bonferroni (conservative) or Holm, or control FDR. Better: use a hierarchical testing plan (test primary first, then second only if primary significant).

**Sequential looks & early stopping**- If you plan interim analyses, adjust alpha via group-sequential methods (O’Brien–Fleming or Pocock) or alpha-spending functions so overall Type I error ≈0.05.- Pre-specify timing (e.g., after 50%, 100% of planned sample) and stopping boundaries for efficacy and futility.- Consider Bayesian/continuous monitoring approaches (credible intervals, decision thresholds) which simplify continuous looks but require different interpretation.- Always pre-register analysis plan and stopping rules to avoid p-hacking.

**UX-specific considerations**- Combine quantitative A/B results with qualitative usability checks to understand why the change worked or not.- If sample budgets are tight, prioritize experiments for highest-impact flows, or run smaller qualitative pilots first.

Unlock Full Question Bank

Get access to hundreds of User Research and Problem Framing interview questions and detailed answers.

Join thousands of developers preparing for their dream job.