1) Approach: three-arm experiment (A/B/C) with equal allocation and a control if desired. Allocation: For primary test only, allocate 25% control, 25% variant A, 25% B, 25% C. If control not needed, 33% each.2) Metrics:- Primary: paid conversion rate (landing → purchase) during experiment window.- Secondary: click-through-to-purchase, time-to-convert, average order value, and post-conversion retention.3) Statistical approach: pre-register primary metric and comparison plan. Use one-way ANOVA or logistic regression to test overall differences, then pairwise comparisons with correction for multiple comparisons (Holm-Bonferroni or Benjamini-Hochberg to control False Discovery Rate). Report effect sizes and CIs.4) Minimum Detectable Effect (MDE): choose MDE based on business significance. Example: baseline conversion = 5%; target MDE = 10% relative lift → absolute MDE = 0.5 percentage points. Compute sample size per arm for desired power (80%) and alpha (0.05 adjusted for multiple comparisons). Example calculation in Python to compute sample size:python
from statsmodels.stats.power import NormalIndPower
power = 0.8
alpha = 0.05/3 # Bonferroni correction for 3 comparisons
baseline = 0.05
mde = 0.005
effect = (mde) / (baseline*(1-baseline))**0.5
analysis = NormalIndPower()
n_per_group = analysis.solve_power(effect_size= mde/((baseline*(1-baseline))**0.5), power=power, alpha=alpha, alternative='two-sided')
print(int(n_per_group))
