Usability Testing and User Research Questions

Focuses on planning, designing, conducting, and analyzing usability studies and broader user research to surface user needs, pain points, and behavior. Candidates should know how to define research goals and questions, design tasks, recruit representative participants, choose between formative and summative approaches and moderated and unmoderated sessions, and select appropriate test formats such as in home, lab, or remote testing. Includes facilitation techniques, observation and note taking, qualitative and quantitative analysis, synthesis of findings into actionable design recommendations, and how to cite user quotes or data to support product decisions.

MediumTechnical

53 practiced

Critique the following planned usability study and propose a better design: 'Three unmoderated tasks, 5 participants, no screener, testing a complex B2B workflow that requires admin privileges.' Identify at least four problems and provide concrete alternatives for method, sample, and controls.

Sample Answer

Problem statementThis plan has multiple fatal flaws for a complex B2B admin workflow. I’d redesign it to get valid, actionable insights.

Key problems (≥4)- Sample size too small: 5 participants won’t capture task variation for complex workflows.- No screener: risk of recruiting non-admins or novices; results won’t generalize to target users.- Unmoderated for complex, privileged tasks: participants can’t be given access, prompted, or debriefed on errors.- Three tasks unspecified: may not cover key success/failure paths or error conditions.- Security/admin-privilege barrier: test environment not addressed; participants may be blocked.

Concrete alternatives — Method- Use remote moderated sessions for discovery and first usability rounds (60–90 min) so you can observe thinking-aloud, unblock permission issues, and probe errors.- For scale validation later, run a follow-up unmoderated prototype test if the flow can be sandboxed safely.

Concrete alternatives — Sample & recruiting- Screen for actual admin users (job role + frequency + tool experience). Example screener: “Have admin privileges in product X?” + “Performed workflow Y in last 3 months?”- Recruit 12–15 participants for moderated testing to reach thematic saturation for complex tasks; 20–30 for unmoderated quantitative validation.- Stratify by experience (junior, senior admins) to see differences in mental models.

Concrete alternatives — Controls & test environment- Provide a dedicated sandbox account with realistic data and admin rights, instrumented with logging.- Create task scenarios with clear success criteria and contingency steps if participants get blocked.- Use session recording + network logs + think-aloud; include post-task interview questions assessing confidence, pain points, and perceived risk.- Add attention/comprehension checks and standardize pre-test training only if that reflects real onboarding.

Metrics & outputs- Qualitative: task success, time-on-task, error types, decision rationale.- Quantitative (if scaled): completion rate, critical error rate, System Usability Scale.- Deliverables: prioritized usability issues mapped to severity and remediation suggestions.

Why this is betterModeration + targeted screener + realistic sandboxing ensures you test the right users, observe root causes of breakdowns, and generate actionable fixes rather than noisy or infeasible findings.

MediumTechnical

53 practiced

You need to triangulate findings across product analytics, in-app surveys, and usability observations to test the hypothesis: 'Users abandon checkout due to price shock.' Describe which signals you'd extract from each data source, how you'd align them by user/cohort, and how you'd handle inconsistent evidence.

Sample Answer

**Approach overview** I’d triangulate quantitative behavioral signals, self-reported reasons, and observed behaviours to validate “price shock” as the cause of checkout abandonment.

**Signals to extract**

- Product analytics - Funnel drop-off rates at price-reveal steps (cart → shipping → payment) and time-on-step - Event counts for promo/discount clicks, price-view events, coupon attempts - Average order value vs. expected price; segmentation by device, geography, referral - Cohort retention and conversion lift after discounts

- In-app surveys - Close-coded reasons (e.g., “too expensive”) and open-text responses - When surveys triggered (after abandonment vs. mid-flow) and response timestamps - Confidence/importance ratings and NPS if available

- Usability observations - Qualitative notes: facial expressions, verbal “that’s higher than I expected”, hesitation, scrolling back to price - Screen recordings showing price discovery moments, misinterpreted fees, sticker shock instances - Frequency and severity counts across participants

**Alignment by user/cohort**- Join by deterministic identifiers when available (user_id, session_id); otherwise align by cohort (traffic source, country, device, A/B group) and time-window.- Create a triangulation table per cohort: analytics metrics + percent citing price in surveys + count of observed price-shock behaviors.- Use time alignment (same day/experiment) and tag users who both responded to surveys and were observed.

**Handling inconsistent evidence**- Assess measurement gaps (sampling bias, low survey response, lab vs. field conditions).- Prioritize evidence: consistent signal across two sources is suggestive; only analytics without qualitative support = hypothesis weak.- Run targeted follow-ups: instrument a micro-survey triggered at price reveal, A/B test pricing display (anchor, fee transparency), and moderated sessions focused on price reveal.- Report confidence levels and recommended next steps (quick experiments vs. deeper ethnography).

EasyTechnical

61 practiced

Explain step-by-step how you would run a quick hallway (guerrilla) usability test inside your company for a prototype: include participant selection, consent, task length, observation method, deliverables, and limitations for making product decisions from this method.

EasyTechnical

67 practiced

Describe the System Usability Scale (SUS): what it measures, how raw scores are converted to the 0-100 scale, typical thresholds for interpreting scores (good/fair/poor), and one limitation of relying solely on SUS to evaluate a prototype.

Sample Answer

**What SUS measures**The System Usability Scale (SUS) is a 10‑item standardized questionnaire capturing perceived usability and learnability of a product. As a design researcher I use it to benchmark user sentiment across designs and track improvements over iterations.

**How to convert raw scores**- Score each odd item: response − 1- Score each even item: 5 − response- Sum the 10 adjusted scores to get raw total (0–40)- Multiply by 2.5 to convert to 0–100

text

raw_total = sum(adjusted_item_scores)   # 0..40
sus_score = raw_total * 2.5             # 0..100

**Typical interpretation thresholds**- ≥ 80.3 — “Excellent” / A grade (high usability)- 68 — “Average” benchmark- 51–67 — “OK / Fair”- ≤ 50 — “Poor” (likely usability problems)

(These thresholds come from industry benchmarking studies and Sauro/Lewis guidelines.)

**One limitation**SUS gives a reliable, single-number snapshot but lacks diagnostic detail: it doesn’t reveal specific task pain points, why users struggle, or which features cause issues. I always pair SUS with qualitative usability testing (think‑aloud, task metrics) to surface actionable insights.

MediumTechnical

62 practiced

Explain, step-by-step, how you would analyze qualitative data from 12 moderated usability sessions. Include approaches for coding (open/axial), tools for affinity mapping, how to identify themes and quantify frequency, and how to validate themes with teammates.

Unlock Full Question Bank

Get access to hundreds of Usability Testing and User Research interview questions and detailed answers.

Join thousands of developers preparing for their dream job.