Growth & Business Optimization Topics
Growth strategies, experimentation frameworks, and business optimization. Includes A/B testing, conversion optimization, and growth playbooks.
Growth and Product Metrics Analysis
Analysis skills specific to growth and product contexts: interpreting funnel metrics, cohort and retention analyses, attribution of acquisition versus activation, detecting seasonality and external event impacts, and diagnosing conversion or engagement changes. Candidates should be able to form hypotheses about what drove changes, propose targeted follow up analyses or A B tests, and identify which additional metrics are needed to evaluate unit economics and growth efficiency.
Feature Success Measurement
Focuses on measuring the impact of a single feature or product change. Key skills include defining a primary success metric, selecting secondary and guardrail metrics to detect negative side effects, planning measurement windows that account for ramp up and stabilization, segmenting users to detect differential impacts, designing experiments or observational analyses, and creating dashboards and reports for monitoring. Also covers rollout strategies, conversion and funnel metrics related to the feature, and criteria for declaring success or rollback.
Segmentation & Dimensionality in Metrics
Learn to think about how metrics vary across dimensions: user segment, geography, traffic source, device type, etc. Practice deciding which dimensions are critical to track separately. Understand why slicing metrics reveals insights (e.g., desktop vs. mobile retention may differ significantly).
Statistical Rigor & Avoiding Common Pitfalls
Demonstrate deep understanding of statistical concepts: power analysis, sample size calculation, significance levels, confidence intervals, effect sizes, Type I and II errors. Discuss common mistakes in test interpretation: peeking bias (checking results too early), multiple comparison problem, regression to the mean, selection bias, and Simpson's Paradox. Discuss how you've implemented safeguards against these pitfalls in your testing processes. Provide examples of times you've caught flawed analyses or avoided incorrect conclusions.
Funnel Analysis and Conversion Tracking
Product analytics practice focused on analyzing user journeys and measuring how well a product or website converts visitors into desired outcomes. Core skills include defining macro and micro conversions, mapping multi step user journeys, designing and instrumenting event level tracking, building and interpreting conversion funnels, calculating step by step conversion rates and drop off, and quantifying funnel leakage. Candidates should be able to segment funnels by cohort, acquisition source, channel, device, geography, or user persona; perform retention and cohort analysis; reason about time based attribution and multi path journeys; and estimate the impact of optimization levers. Practical competencies include implementing tracking, validating data quality, identifying common pitfalls such as missing events or incorrect attribution windows, and using split testing and iterative analysis to validate hypotheses. Candidates should also be able to diagnose root causes of drop off, create mental models of user behavior, run diagnostic analyses and experiments, and recommend prioritized interventions and product or experience changes with expected outcomes and measurement plans.
Impact Driven Mindset
Approach and habits that prioritize measurable impact over activity. Topics include defining success criteria and hypotheses, using data to compare and prioritize initiatives, selecting work with the highest expected business return, balancing short term wins and long term investments, time boxing experiments and minimum viable solutions to learn quickly, and communicating impact oriented choices to stakeholders. Candidates should be ready to show examples of how they set impact goals, measured results, and redirected effort based on outcomes.
Marketing Funnel Optimization and Experimentation
Using data and controlled experimentation to improve conversion across marketing funnels. Subjects include mapping the funnel and instrumenting events and metrics for each stage, identifying bottlenecks, framing hypotheses and prioritizing experiments by impact, designing and running A B tests and controlled experiments with appropriate sample size considerations, segment and personalization strategies, post experiment analysis including effect size and significance interpretation, attribution considerations, and implementing technical or workflow changes to capture gains. Candidates should be able to describe concrete experiments, metric definitions, and how findings influenced product or marketing decisions.
Experimentation Methodology and Rigor
Focuses on rigorous experimental methodology and advanced testing approaches needed to produce reliable, actionable results. Topics include statistical power and minimum detectable effect trade offs, multiple hypothesis correction, sequential and interim analysis, variance reduction techniques, heterogenous treatment effects, interference and network effects, bias in online experiments, two stage or multi component testing, multivariate designs, experiment velocity versus validity trade offs, and methods to measure business impact beyond proximal metrics. Senior level discussion includes designing frameworks and practices to ensure methodological rigor across teams and examples of how to balance rapid iteration with safeguards to avoid false positives.
A and B Test Design
Designing and running A and B tests and split tests to evaluate product and feature changes. Candidates should be able to form clear null and alternative hypotheses, select appropriate primary metrics and guardrail metrics that reflect both product goals and user safety, choose randomization and assignment strategies, and calculate sample size and test duration using power analysis and minimum detectable effect reasoning. They should understand applied statistical analysis concepts including p values confidence intervals one tailed and two tailed tests sequential monitoring and stopping rules and corrections for multiple comparisons. Practical abilities include diagnosing inconclusive or noisy experiments detecting and mitigating common biases such as peeking selection bias novelty effects seasonality instrumentation errors and network interference and deciding when experiments are appropriate versus alternative evaluation methods. Senior candidates should reason about trade offs between speed and statistical rigor plan safe rollouts and ramping define rollback plans and communicate uncertainty and business implications to technical and non technical stakeholders. For developer facing products candidates should also consider constraints such as small populations cross team effects ethical concerns and special instrumentation needs.