Experimentation & Growth Metrics Topics
Growth strategies, experimentation frameworks, and business optimization. Includes A/B testing, conversion optimization, and growth playbooks.
Segmentation Scheme Design and Governance
Designing and governing the segmentation schemes used to analyze product and business metrics. Covers choosing which dimensions materially affect a metric, cardinality handling and the bucketing of continuous dimensions, correlated and redundant dimensions, RFM versus clustering versus rule-based segmentation, granularity against statistical power, and canonical segment definitions that stay stable across analyses. The scope is how a segmentation is built, chosen and maintained, not diagnosing why a specific metric moved.
Attribution & Conversion Measurement
Measuring what drives a conversion: event tracking and instrumentation, attribution models (first-touch, last-touch, multi-touch), and connecting user actions to outcomes. Covers the analytics plumbing for reliable conversion measurement and the limits of each attribution approach. The scope is the measurement layer for conversions, not the creative or channel strategy that generates them.
A/B Test Design & Statistical Rigor
Designing and statistically defending a controlled online experiment: framing a testable hypothesis, defining control and treatment variants, choosing the randomization unit, setting the primary success metric, and computing sample size, power, and minimum detectable effect. Covers the statistical foundations that make a readout trustworthy, including hypothesis testing, p-values, confidence intervals, statistical vs practical significance, and Type I/II error. Emphasizes avoiding the common pitfalls that invalidate a test, such as peeking, multiple-comparison inflation, underpowered designs, and how test duration and stopping rules affect the validity of conclusions.
Conversion Funnel Optimization
Analyzing and improving a bounded, ordered conversion path: mapping the sequence of steps a user takes from acquisition through one terminal conversion or activation event (signup, first purchase, first paid order, trial-to-paid, onboarding to first-success), computing step-to-step and overall conversion rates and drop-off, and diagnosing where and why users fall out. Covers the SQL and query techniques for computing funnel metrics at scale (stage-by-stage conversion tables, time-to-conversion and time-to-first-value, cohort LTV measured within a funnel window, path analysis across non-linear user journeys, event instrumentation and data-quality practices for funnel tracking), attribution modeling for crediting conversions across channels and touchpoints (first-touch, last-touch, linear, time-decay, Markov-chain, and Shapley-value approaches) and customer acquisition cost by channel, and the experiment design and statistics used to validate funnel changes (A/B and multi-armed-bandit test design, sample-size and power calculations, quasi-experimental methods such as difference-in-differences and synthetic control when randomization is not possible, and testing whether a single funnel-stage drop is a real, statistically significant shift rather than noise). Also covers diagnosing UX and flow friction that causes drop-off (checkout, signup, and onboarding friction points) and prioritizing a program of funnel-improvement experiments (impact and effort frameworks such as RICE or ICE, guardrail metrics, roadmap sequencing). Distinct from User Retention and Engagement, which covers what an already-converted or already-activated user does afterward: repeat usage over time, cohort retention curves, DAU/WAU/MAU, churn, and reactivation. A question belongs here if it concerns a user's first, bounded pass toward one conversion or activation event; it belongs to User Retention and Engagement if it concerns recurring behavior after that event. General-purpose rolling-window anomaly and change-point detection techniques (CUSUM, Bayesian change-point, seasonality-aware baselines) for monitoring any metric over time belong to the companion topic Advanced SQL: Metric Monitoring, Anomaly Detection, and Data Correctness at Scale, not here.