Experimentation & Growth Metrics Topics
Growth strategies, experimentation frameworks, and business optimization. Includes A/B testing, conversion optimization, and growth playbooks.
Experiment Analysis & Result Interpretation
Reading out an experiment after it runs: interpreting the treatment effect, deciding ship/no-ship, and reconciling conflicting or flat results. Covers reasoning under uncertainty, acting on inconclusive or limited data, and translating a measured effect into a business decision. The emphasis is turning experiment output into a defensible recommendation.
A/B Test Design & Statistical Rigor
Designing and statistically defending a controlled online experiment: framing a testable hypothesis, defining control and treatment variants, choosing the randomization unit, setting the primary success metric, and computing sample size, power, and minimum detectable effect. Covers the statistical foundations that make a readout trustworthy, including hypothesis testing, p-values, confidence intervals, statistical vs practical significance, and Type I/II error. Emphasizes avoiding the common pitfalls that invalidate a test, such as peeking, multiple-comparison inflation, underpowered designs, and how test duration and stopping rules affect the validity of conclusions.
Product-Led Growth & Self-Serve Funnels
Growth driven by the product itself: self-serve signup and onboarding funnels, free-to-paid conversion, and in-product mechanics that acquire and expand users without sales touch. Covers instrumenting and optimizing the self-serve journey and the metrics that gauge a product-led motion. The concept scope is the PLG model and its funnels.
Metric Frameworks & Guardrails
Designing a coherent metric framework to steer a product: defining a north-star metric, building metric hierarchies, distinguishing leading from lagging indicators, and aligning metrics to goals so that what is measured drives the intended behavior while avoiding vanity or easily-gamed metrics. Covers the guardrail side of the same design work: defining guardrail metrics, detecting negative side effects, and reasoning about tensions between competing metrics where a win on one degrades another. The scope is choosing, structuring, and safeguarding a metric system, not diagnosing a specific movement.
Metric Diagnosis & Segmentation Analysis
Investigating why a metric moved: root-causing a spike, drop, or plateau by decomposing it across segments and dimensions. Covers segmentation, cohorting, Simpson's-paradox traps, and distinguishing a real change from seasonality or a tracking artifact. The scope is diagnostic metric analysis rather than choosing which metric to track.
Conversion Funnel Optimization
Analyzing and improving a multi-step conversion funnel: mapping the journey, quantifying drop-off at each stage, and diagnosing where and why users fall out. Covers conversion-rate optimization tactics, landing-page and flow improvements, and structuring an optimization program. The concept scope is funnel analytics and the levers that raise stage-to-stage conversion.
Growth Metrics & Unit Economics
The core quantitative vocabulary of growth: activation, retention, referral and revenue metrics, growth-accounting frameworks such as AARRR, and unit economics including LTV, CAC, and payback period. Covers defining these metrics precisely and computing growth calculations that reveal whether growth is efficient and durable. The scope is metric definitions and economic math, not go-to-market execution.
User Retention & Engagement
Measuring and improving how users stick: retention curves, cohort retention, engagement frequency and depth, and lifecycle stages from onboarding to resurrection and churn. Covers diagnosing where retention breaks and the interventions that deepen habitual usage. The concept scope is the retention and engagement side of the user lifecycle.
Feature Success Measurement
Judging whether a shipped feature worked: defining success criteria before launch, measuring adoption and impact, and separating a feature's effect from background trends. Covers post-launch readouts, tying a feature to a target metric, and deciding whether to iterate, keep, or roll back. The scope is evaluating feature impact rather than designing the test that produced it.