Research Planning and Fieldwork Questions

Approach summary
I prioritize the backlog with a transparent scoring model that combines impact, confidence, effort, and risk, and explicitly propagates uncertainty from research into priority scores. I report ranges not single points so stakeholders see robustness.

Scoring model

• Score = (Expected Impact × Confidence) / (Effort × (1 + Risk))
  • Expected Impact: mean customer value (0–10)
  • Confidence: 0–1 multiplier derived from evidence quality
  • Effort: story points normalized (min 1)
  • Risk: 0–1 representing technical/ops/regulatory risk

Example: Impact 8, Confidence 0.6, Effort 5, Risk 0.2 → Score = (8×0.6)/(5×1.2)=4.8/6=0.8

Quantifying uncertainty for qualitative findings

• Convert qualitative insights to distributions: e.g., “majority prefer X” → Impact ~ Normal(mu=7, sd=1.5); “mixed feedback” → wider sd (sd=2.5).
• Map evidence strength to Confidence: user interview N<5 → 0.4; usability test N=20 with clear effect → 0.8.
• Use Bayesian updating: prior impact from market/benchmarks, update with new study to produce posterior mean & sd.

Sensitivity analysis

• Monte Carlo simulation: sample Impact and Confidence from their distributions 10k times, compute score distribution per backlog item.
• Tornado chart: vary one input (±25–50%) to see which drives rank changes.
• Threshold analysis: identify breakpoints where item A overtakes B (e.g., if Confidence_A < 0.45, B ranks higher).
• Present results: median score, 10–90th percentile, probability each item is top-3. Use this to surface high-uncertainty candidates for experiments.

Stakeholder use

• Use visuals (rank distributions, tornado, probability table) to discuss trade-offs and decide whether to build, prototype, or run targeted research to reduce uncertainty.

Approach / goal
Briefly reconcile why qualitative interviews say users are “confused” about Feature A while analytics show higher time-on-site and engagement. Goal: determine whether engagement is positive (deeper value) or caused by friction/confusion.

Method (triangulation steps)

1. Revisit definitions & metrics

   • Clarify what “confused” means in interviews (task failure, hesitation, wrong mental model).
   • List analytics metrics: time-on-site, clicks, conversion, drop-off, scroll depth, session replay rates.

2. Run targeted mixed-method tests

   • Moderated usability sessions focused on the tasks using Feature A (observe paths, ask think-aloud).
   • Instrument funnel analytics and add event logging for key success/error events.
   • Use session replays + heatmaps to see if increased time equals exploration or getting stuck.

3. Quantify and compare

   • Code qualitative themes into frequencies (e.g., 40% said unclear CTA).
   • Correlate interview themes with behaviors (users who hesitated → longer sessions, lower conversion).

Weighing data & surfacing uncertainty

• Prioritize direct behavioral evidence: observed user errors + conversion impact > aggregate time metric.
• Treat analytics as high-confidence but ambiguous; treat qualitative as lower sample-size but high-fidelity for intent.
• Present confidence levels (High/Medium/Low) and sample sizes; show alternative explanations and p-values or effect sizes where available.

Recommendation to product team

• Quick fixes: clarify label/affordance based on verbatim quotes + A/B test copy/CTA.
• Monitor: implement event logging and A/B test; target metric: reduce time-to-complete task by X% while maintaining engagement.
• Next steps: 5 moderated sessions post-change + 4-week A/B test. If engagement drops but conversion rises, iterate on discoverability patterns.

This balances quantitative scale with qualitative insight and gives a concrete test-and-measure path.

Clarify goals & constraints (month 0)

• Goal: scale lightweight, repeatable discovery across 8 product teams; reduce decision latency; raise research coverage to 80% of major bets.
• Constraints: budget, privacy regs (GDPR/CCPA), internal IRB, 2 FTE research-ops hire planned.

12‑Month Roadmap (quarterly milestones)

Q1 — Foundation

• Participant program: audit existing studies, define target segments, pilot centralized participant panel (100 profiles).
• KM: create research repo taxonomy in Notion/Confluence; publish 3 templates (recruit screener, discussion guide, synthesis).
• Tooling: shortlist tools (Dovetail, Lookback/Hotjar, Typeform, UserZoom); run 30-day trials.
• Governance: draft consent language, data retention policy; legal review.
• Training: 2-hour intro workshop for PMs/eng on when/how to recruit researchers.
• Costing: estimate CAPEX/OPEX; submit budget for tools + 1 ops FTE.
• KPI: repo uptake, panel sign-ups, trainings held.

Q2 — Scale & Automate

• Participant program: expand panel to 500, implement incentives, build scheduling automation (Calendly + Zapier).
• KM: migrate past artifacts into repo; implement tagging and readme.
• Tooling: finalize contracts; enable session replay and moderated remote testing; integrate survey pipelines.
• Governance: implement consent capture flows; anonymization guidance.
• Training: hands-on moderation clinic; office hours.
• Costing: negotiate volume discounts.
• KPI: study throughput, time-to-insight, panel response rates.

Q3 — Embed in Teams

• Participant program: cross-team SLAs for recruitment; create quick-recruit channels.
• KM: templates + playbook; UX pattern library links.
• Tooling: integrate tools with product analytics and ticketing.
• Governance: regular audits; consent dashboard.
• Training: role-based micro-courses for PMs/engineers; shadowing program.
• KPI: percent of projects with discovery, reduction in build rework.

Q4 — Optimize & Measure ROI

• Program: maintain panel quality; cohort targeting for longitudinal studies.
• KM: analytics on repo usage; refine search.
• Tooling: instrument session replay for key flows; A/B research experiments.
• Governance: external compliance review.
• Training: train-the-trainer; embed research checklist in sprint rituals.
• Costing: ROI report (cost per insight, impact on NPS/activation).
• KPI: insight-to-action time, adoption rate, ROI metrics.

Governance & Consent

• Standard consent scripts, opt-in panel portal, centralized data retention and deletion workflows, quarterly reviews with legal.

Training

• Curricula: discovery 101, writing hypotheses, moderating basics, interpreting qualitative data for engineers/PMs.
• Format: micro-modules, hands-on labs, office hours, certification.

Costing approach

• Line items: panel incentives, tools (licenses), 1–2 ops FTEs, training hours.
• Build a 12-month P&L and run sensitivity scenarios (low/high adoption).

KPIs (example)

• Operational: studies/month, average recruitment time, panel response rate.
• Adoption: % of teams using repo/templates, % projects with discovery.
• Impact: reduction in rework, time-to-decision, qualitative insight-to-product changes.
• Quality: participant diversity score, consent compliance rate.

Why this works: incremental roll‑out balances speed and governance, embeds research into team rituals, and ties ops to measurable business impact so research scales sustainably.

Overview
As a product designer I treat accessibility as a research constraint that shapes sampling, methods, tasks, testing, and reporting so outcomes reflect real-world use.

Participant recruitment

• Define inclusion criteria by functional needs (vision, hearing, mobility, cognitive) not just medical labels.
• Recruit via disability orgs, accessibility panels, and compensation that covers extra time/assistive tech.
• Screen for assistive tech used (screen readers, magnifiers, switch devices).

Method selection & task design

• Prefer mixed methods: moderated sessions, remote unmoderated with assistive-tech support, and contextual inquiries.
• Design tasks that mirror real goals, allow flexible workflows, and avoid time limits that penalize slower input.
• Provide alternative response modes (voice, typed, observational notes).

Assistive-technology testing

• Test with participants’ own devices/configurations; include synthesized speech and mobile magnification.
• Log settings, shortcuts, and workarounds; run compatibility checks with major AT (NVDA, VoiceOver, TalkBack).

Reporting & design translation

• Report personas by capability, cite concrete failure modes and reproducible reproduction steps.
• Prioritize fixes by severity and frequency; include AT-specific acceptance criteria in tickets and components in the design system.

Concrete steps I’d take

1. Build a recruitment screener capturing AT and environment.
2. Run pilot sessions with 2–3 AT users to refine tasks.
3. Conduct 8–12 mixed-method sessions with balanced representation.
4. Synthesize into accessibility insights, provide annotated designs, and create actionable tickets with acceptance criteria and test scripts.

Brief approach (from a Product Designer lens)
I create personas by synthesizing qualitative + quantitative research into a compact, actionable profile that informs decisions across design and product.

Essential persona fields

• Name + photo (representative)
• Snapshot: role, age-range, location, tech context
• Goals & motivations (top 3)
• Key pain points / frustrations (top 3)
• Behaviors & workflows (when/how they use product)
• Tech proficiency & constraints
• Representative quote (verbatim from research)
• Critical scenarios / job-stories (see example)
• Success metrics (how we’ll know they win)
• Research sources & confidence level

Example job-story:
“When [situation] I want to [motivation] so I can [expected outcome].”
“When my team misses a deadline, I want a clear task summary so I can prioritize fixes before the next release.”

Common pitfalls

• Creating fictional stereotypes without research
• Overpacking personas with irrelevant detail
• Too many personas (diffuses focus)
• Keeping personas as static PDFs nobody references
• Using personas instead of real user excerpts

Keep personas actionable

• Tie each persona to 1–2 prioritized job-stories and acceptance criteria
• Link persona fields to raw research and session clips in a living doc
• Use personas in design crits, roadmap prioritization, and user-journey mapping
• Create measurable success metrics per persona (adoption, task time, NPS)
• Run regular “persona sanity” sessions after new research; update confidence levels
• Embed personas in templates (PRDs, user flows, A/B test hypotheses)

This keeps personas evidence-based, decision-focused, and integrated into day-to-day design and PM work.