Experimentation Roadmap and Learning Questions

Covers how individual experiments and tests are planned and sequenced within a broader experimentation program to generate reliable, actionable learning. Topics include hypothesis formulation, experimental design, metrics and success criteria, prioritization frameworks, resource allocation, and sequencing to enable sequential learning and knowledge accumulation. Also covers managing a portfolio of experiments, deciding when to stop or scale treatments, instrumentation and data quality considerations, handling confounders and bias, documenting results, and ensuring learnings are transferred into product and business decisions. Candidates should be able to explain trade offs between speed and statistical rigor, how to structure experiments to maximize organizational learning, and how to align experiments to strategic objectives.