Research Design and Study Planning Questions

End to end planning and design of research studies to rigorously answer product, user experience, or scientific questions. Candidates should be able to translate business or product problems into clear and testable research questions and hypotheses and convert those questions into feasible and valid study plans. Core skills include selecting appropriate qualitative, quantitative, or mixed methods, defining primary outcomes and success metrics, aligning sampling strategy and inclusion and exclusion criteria, estimating sample sizes and articulating precision and power considerations, designing recruitment approaches and consent procedures, drafting interview guides survey items and measurement instruments with attention to reliability and validity, planning data collection workflows and quality controls, and outlining statistical and qualitative analysis plans and integration strategies for mixed methods. Candidates should also be able to identify potential confounds and threats to internal and external validity and propose mitigation approaches, scope studies to remain feasible under time and resource constraints, plan logistics timelines and resource allocation, pilot and iterate instruments, address ethical and regulatory requirements such as institutional review board review and data privacy, and communicate research plans limitations and actionable findings to stakeholders. Interviewers may probe trade offs among methodologies bias mitigation strategies reproducibility and documentation practices how the candidate managed scope and stakeholder expectations and how preliminary findings or stakeholder input influenced the evolution of research questions and study scope while avoiding scope creep.

EasyTechnical

57 practiced

List and briefly compare five recruitment approaches for remote user research (for example: panel vendors, intercept, customer mailing list, social ads, in-app prompts). For each approach note typical cost, expected speed to recruit, and common quality trade-offs.

EasyTechnical

84 practiced

Describe three common sampling strategies used in product research (for example: convenience, stratified, random). For each strategy provide: a one-sentence definition, two typical use-cases in product work, and one limitation relevant to product decision-making.

MediumTechnical

43 practiced

Estimate required sample size for an A/B test where baseline conversion is 5%, desired minimum detectable effect (MDE) is an absolute 1.5 percentage points, power = 0.8, and alpha = 0.05. Explain the steps and formulas you would use (normal approximation, pooled proportions) and provide an approximate per-group sample size estimate or the logic to compute it.

Sample Answer

**Answer (design-researcher framing)**

**1) Clarify & numbers**- Baseline p1 = 0.05, desired p2 = 0.065 (absolute MDE = 0.015)- Alpha = 0.05 (two-sided) → z = 1.96- Power = 0.8 → z = 0.84

**2) Formula (normal approximation, pooled-ish variance)**Use the common two-proportion approximate sample-size per group:

text

n = [ (z_{1-α/2} * sqrt(2 * p_bar * (1 - p_bar))
      + z_{power} * sqrt(p1*(1-p1) + p2*(1-p2)) )^2 ] / (Δ^2)

where p_bar = (p1 + p2)/2 and Δ = p2 - p1.

**3) Plug numbers (steps)**- p_bar = 0.0575- sqrt(2 * p_bar * (1-p_bar)) ≈ 0.3291- sqrt(p1*(1-p1) + p2*(1-p2)) ≈ 0.3291- numerator ≈ (1.96*0.3291 + 0.84*0.3291)^2 ≈ 0.848- Δ^2 = 0.015^2 = 0.000225

text

n ≈ 0.848 / 0.000225 ≈ 3,767 per group

**4) Practical notes**- ~3.8k users per arm (so ~7.6k total). With low baseline rates, sample sizes grow quickly.- If you run one-sided test, or accept lower power, sample needed falls.- Use exact binomial or simulation for small samples or if assumptions (independence, equal traffic) fail.- In design research terms: ensure metrics, unit of randomization, and washout (repeat users) are controlled so the statistical assumptions hold.

HardTechnical

46 practiced

Leadership asks you to deliver actionable insights in 10 business days on a sudden usability regression identified in analytics. Produce a prioritized, time-boxed research plan that balances speed and rigor: specify methods (analytics triage, heuristic review, rapid unmoderated tests, targeted interviews), sampling approach, shortcuts for analysis, and how you will communicate preliminary versus final findings.

Sample Answer

**Overview & Goal (10 business days)** Deliver validated, actionable insights about the usability regression fast enough for a hotfix while preserving rigor for final recommendations.

**Day-by-day, prioritized plan (time-boxed)** - Days 1–2 — Analytics triage (priority 1) - Run funnel/drop-off, event-segmentation, and cohort analyses in GA/Amplitude to localize regression (flow, device, OS, country, version). - Quick KPI outputs: % delta by segment, top 3 suspect screens/events. Deliverable: one-page dashboard + annotated hypothesis list. - Days 3–4 — Heuristic review + reproducibility (priority 2) - Expert review of implicated screens against usability heuristics and platform guidelines. - Attempt to reproduce issues across devices/browsers; log steps and screenshots. Deliverable: annotated screenshots + severity matrix. - Days 5–8 — Rapid user validation (priority 3) - Rapid unmoderated tests (5–8 tasks, 20–30 participants across affected segments; mix mobile/desktop, target top 2 languages). Use Lookback/UsabilityHub/Playbook. Capture task success, time, and videos. - Targeted moderated interviews (3–5 users) for edge cases or to probe motivations where analytics ambiguous. 60-min sessions, focused script. - Sampling: purposive — users from segments showing highest regression; recruit via in-product intercept and panel. - Days 9–10 — Synthesis & recommendations (priority 4) - Rapid affinity mapping, prioritize fixes by impact/effort, propose A/B or rollback if needed. Deliverables: 1-page executive brief (prelim), 6–8 slide stakeholder deck (final) and a prioritized backlog.

**Shortcuts for speed (while preserving rigor)** - Use sampling heuristics: prioritize segments with highest absolute impact and highest strategic value. - Codebook-lite: simple, consistent tags for issues during video review to enable fast aggregation. - Stop rules: if 70% of unmoderated sessions show same failure mode, treat as confirmed. - Leverage analytics segments as ground truth to weight qualitative findings.

**Prelim vs Final communication** - Day 3: “Preliminary alert” email with analytics snapshot and high-confidence hypotheses. - Day 5: Share interim deck with heuristic findings and reproduction logs. - Day 10 (end): Deliver final report with evidence (analytics charts, clips, quotes), prioritized fixes (hotfix vs design backlog), recommended experiment/rollback plan, and metrics to track post-fix. Include an appendix with raw artifacts for engineers.

**Success criteria & next steps** - Clear mapping: identified cause → fix recommendation → expected metric delta. - Proposed monitoring: 48–72h post-deploy analytics watch + follow-up usability check.

HardTechnical

56 practiced

You plan to run a study collecting behavioral and demographic data across the EU and US, including screen recordings. Outline the IRB/ethics and GDPR-related considerations you must address (consent language, legal basis, data transfers, data subject rights) and list practical steps to ensure compliance (e.g., local approvals, Data Protection Impact Assessment, data minimization, data residency).

Sample Answer

**Brief framing (role & scope)** As a design researcher running screen-recording studies across the EU and US, I must satisfy both IRB/ethics and GDPR requirements: protect participant welfare, document informed consent, establish lawful basis for processing, and ensure lawful cross-border transfers.

**IRB / ethics considerations** - Use IRB/ethics board approval that describes risks of recording sensitive on-screen content and mitigation (de-identification, pause/stop controls). - Consent language must explain purpose, types of data (screen, audio, demographics), duration, voluntary participation, right to withdraw, and how findings will be used/published. - Provide researcher contact and escalation path for participant distress.

**GDPR considerations** - Legal basis: obtain explicit consent for special categories or where personal data is processed; alternatively rely on legitimate interest only after balancing test (rare for recordings). - Consent wording: specific, granular (separate consent for recording, audio, retention), easy to withdraw, and recordable (audit trail). - Data subject rights: provide mechanisms for access, rectification, erasure, portability, restriction, and objection; include DS contact and supervisory authority info. - Data transfers: use EU Standard Contractual Clauses or ensure adequacy decision; encrypt in transit and at rest; limit transfers to subprocessors with SCCs and documented DPIA coverage.

**Practical steps to ensure compliance** - Conduct a Data Protection Impact Assessment (DPIA) before the study. - Data minimization: record only needed regions/times; mask PII on recordings; avoid collecting unnecessary demographics. - Data residency & retention: store EU data on EU-resident servers; define minimal retention and secure deletion processes. - Technical controls: pseudonymize identifiers, encrypt, access controls, logging, and secure playback tools for analysts. - Organizational controls: sign DPA with vendors, maintain audit trail of consent, train team on handling recordings, and obtain local approvals where required (national research laws or institutional policies). - Participant workflow: show consent form, demo recording, allow on-the-spot stop/delete, and provide follow-up with data-access/erasure instructions.

**Trade-offs & justification** - Prefer explicit consent and EU-residency to reduce transfer risk; if relying on legitimate interest, document balancing test and mitigation. - DPIA may recommend additional masking or excluding certain user groups to reduce risk.

This approach balances ethical participant care with GDPR compliance while enabling useful behavioral insights for design.

Unlock Full Question Bank

Get access to hundreds of Research Design and Study Planning interview questions and detailed answers.

Join thousands of developers preparing for their dream job.