Goal & risky assumption
Assumption: New onboarding carousel that asks 3 progressive profile questions will increase activation (complete profile → first key action). I need a lightweight test to validate whether users will complete multi-step onboarding.
Step-by-step plan
- Prototype fidelity
- Medium-fidelity clickable prototype (Figma + interactive flows) showing carousel, progress indicator, and skip CTAs. Simulate state changes and success screen; no backend required.
- Recruitment
- 50–100 target users recruited from in-product banners (if available) or an intercept panel (Targeted via existing users segmented by new sign-ups in last 7 days). Offer small incentive ($5 gift card) or in-app perk.
- Experiment flow
- Randomly assign visitors (n≈100) to control (current onboarding) or experiment (carousel). Show prototype via moderated remote test (10–15 sessions) and unmoderated via clickable link for remaining.
- Metrics to track (primary, secondary, qualitative)
- Primary: Activation rate = % who complete profile + perform first key action within 24h.
- Secondary: Completion rate of carousel steps, time to complete, skip rate.
- Qualitative: Usability issues, confusion points from session notes and open feedback.
- Decision rules
- Continue to build if experiment increases activation by ≥10 percentage points with p-value <0.1 (or consistent directional uplift across quantitative + positive qualitative signals).
- Iterate if completion rate is high but activation unchanged — tweak CTA or timing.
- Stop if completion < control by ≥5 points or qualitative feedback shows major confusion.
Why this approach
Minimal engineering, fast turnaround, mixes quantitative and qualitative signals to reduce risk before committing engineering resources.
