Covers fundamental probability theory and statistical inference from first principles to practical applications. Core probability concepts include sample spaces and events, independence, conditional probability, Bayes theorem, expected value, variance, and standard deviation. Reviews common probability distributions such as normal, binomial, Poisson, uniform, and exponential, their parameters, typical use cases, computation of probabilities, and approximation methods. Explains sampling distributions and the Central Limit Theorem and their implications for estimation and confidence intervals. Presents descriptive statistics and data summary measures including mean, median, variance, and standard deviation. Details the hypothesis testing workflow including null and alternative hypotheses, p values, statistical significance, type one and type two errors, power, effect size, and interpretation of results. Reviews commonly used tests and methods and guidance for selection and assumptions checking, including z tests, t tests, chi square tests, analysis of variance, and basic nonparametric alternatives. Emphasizes practical issues such as correlation versus causation, impact of sample size and data quality, assumptions validation, reasoning about rare events and tail risks, and communicating uncertainty. At more advanced levels expect experimental design and interpretation at scale including A B tests, sample size and power calculations, multiple testing and false discovery rate adjustment, and design choices for robust inference in real world systems.
MediumTechnical
51 practiced
You must evaluate a checkout flow UI change on an e-commerce platform with baseline conversion 3% and 2 million daily unique users. Design the A/B test: state the primary metric, how to randomize, pre-specify hypotheses, compute the sample size per variant for detecting a 10% relative lift at 80% power and alpha=0.05 (two-sided), and estimate how long the test will run. Describe bias controls and monitoring strategy.
Sample Answer
**Primary metric**- Daily per-user checkout conversion (binary: completed checkout within session). Use risk ratio and absolute difference as secondary.**Randomization**- Randomize at user-id (cookie or logged-in ID) level, persistent assignment across sessions. 1:1 allocation to Control vs Variant using hashed user-id modulo.**Pre-specified hypotheses**- H0: p_variant = p_control (no change).- H1: p_variant ≠ p_control (two-sided). - Baseline p0 = 0.030. Target relative lift = 10% → p1 = 0.033.**Sample-size calculation (per variant)**Use normal approximation for two-sample proportion test with alpha=0.05 (two-sided), power=0.80, Z_{1-alpha/2}=1.96, Z_{power}=0.84.
Plugging numbers yields approximately n_per_arm ≈ 53,000 users. Total ~106,000 unique users.**Estimated run time**- With 2,000,000 daily unique users and 1:1 split → ~1,000,000 users/arm/day. Statistically sufficient in ~0.053 days (~1.3 hours).- Practical recommendation: run at least 7 full days (to cover weekday/weekend effects, traffic skew, novelty effects) and collect a safety margin (e.g., 2× sample) → ~1–2 days to several weeks depending on business constraints; default operational minimum = 7 days.**Bias controls**- Persistent user assignment to avoid crossover.- Exclude bot traffic and instrumentation-only cohorts.- Ensure tracking parity: identical event definitions and instrumentation across variants.- Block randomization by device/region if large imbalance expected; or stratify and analyze heterogeneity.**Monitoring & stopping rules**- Pre-specify analysis plan, primary metric, and no-peeking policy; prefer fixed-horizon test. If using sequential monitoring, apply alpha-spending (e.g., O’Brien–Fleming).- Real-time dashboards: conversion, sample counts, balance checks (covariates by arm), click events, errors, latency, checkout drop-off points.- Safety alerts for adverse signals (significant revenue loss, error rate spike). If operational issues occur, pause and investigate.This design balances statistical rigor with practical safeguards appropriate for an Applied Scientist deploying experiments at scale.
MediumTechnical
51 practiced
In a linear regression model predicting revenue per user, explain the difference between a confidence interval for the mean response at a particular x* and a prediction interval for a new observation at x*. Provide formulas (assuming homoscedastic Gaussian errors) and describe how interval widths change with sample size and residual variance.
Sample Answer
**Answer (Applied Scientist perspective)****Setup / assumptions**- Simple linear regression: y = β0 + β1 x + ε, with ε ~ N(0, σ^2) (homoscedastic).- Let ŷ(x*) = β̂0 + β̂1 x*; s^2 = RSS/(n-2); Sxx = Σ (xi − x̄)^2.**Confidence interval for the mean response at x\***
Plain English: interval for E[y | x*] — uncertainty comes from estimating β̂0, β̂1 (no observation noise).**Prediction interval for a new observation at x\***
Plain English: interval for a single future user’s revenue at x* — includes both parameter uncertainty and irreducible residual variance σ^2 (hence the extra +1 term inside sqrt).**How widths change**- Residual variance (s^2): both intervals scale linearly with s; larger s → wider intervals.- Sample size (n): increasing n reduces the 1/n term and typically reduces s (better estimates), so both intervals shrink; CI for mean shrinks faster because it lacks the +1 irreducible term.- Location relative to x̄: intervals widen as x* moves away from x̄ because (x*−x̄)^2/Sxx grows.- Always: PI > CI at same x* because of the extra +1 (irreducible noise).**Practical note**- If errors are heteroscedastic, these formulas are invalid; use weighted least squares or heteroscedasticity-robust prediction methods (e.g., bootstrap or model the variance).
MediumTechnical
69 practiced
Compare Bonferroni correction and the Benjamini-Hochberg (BH) procedure for multiple testing. Given p-values [0.001, 0.02, 0.03, 0.2] (m=4) and alpha=0.05, identify which hypotheses are rejected by Bonferroni and by BH. Explain when each method is preferable in an applied-science setting.
Sample Answer
**Brief comparison**- Bonferroni controls the family-wise error rate (FWER): probability of any false positive ≤ α. It's conservative: divides α by m.- Benjamini–Hochberg (BH) controls the false discovery rate (FDR): expected proportion of false discoveries among rejected hypotheses. Less conservative, more power for many tests.**Apply to p = [0.001, 0.02, 0.03, 0.2], m = 4, α = 0.05**Bonferroni:- Adjusted threshold = α / m = 0.05 / 4 = 0.0125- Reject if p ≤ 0.0125 → only 0.001 is rejected.Result: reject 1 hypothesis (p = 0.001).Benjamini–Hochberg:- Sort p and compute thresholds (i/m)*α: - i=1: 0.0125 (0.001 ≤ 0.0125) - i=2: 0.025 (0.02 ≤ 0.025) - i=3: 0.0375 (0.03 ≤ 0.0375) - i=4: 0.05 (0.2 > 0.05)- Largest i with p(i) ≤ threshold is i = 3 → reject the first three smallest p-values.Result: reject p = 0.001, 0.02, 0.03.**When to prefer each (applied-science perspective)**- Use Bonferroni when any false positive is costly (clinical trials, regulatory decisions) or when m is small. It guarantees strong control of even a single Type I error.- Use BH when you run many parallel tests and want higher power while tolerating some controlled proportion of false discoveries (genomics, A/B testing of many variants, feature screening). BH is typical in exploratory analyses or when downstream validation can filter false positives.**Practical notes**- BH assumes independent or positively dependent tests for exact FDR control; under arbitrary dependence it’s still often used but may be slightly anti-conservative—alternatives (e.g., Benjamini–Yekutieli) exist.- In applied work, consider follow-up validation for BH-identified hits to confirm practical significance.
EasyTechnical
49 practiced
Explain the Central Limit Theorem (CLT) and its practical implications for building confidence intervals for sample means. In particular, discuss why practitioners often use CLT-based normal approximations for n around 30, what assumptions are required, and give an example of when n=30 might not be sufficient.
Sample Answer
**Brief definition & intuition**The Central Limit Theorem (CLT) states that for i.i.d. random variables X1...Xn with finite mean μ and finite variance σ^2, the sampling distribution of the standardized sample mean
text
( X̄ - μ ) / ( σ / sqrt(n) )
converges to a standard normal as n → ∞. Intuitively, aggregation of many independent effects washes out individual idiosyncrasies.**Practical implication for CIs**- For large n, X̄ ≈ Normal(μ, σ^2/n). Construct CI using known σ: - X̄ ± z_{α/2} * σ/√n- If σ unknown (typical), use sample s and t-distribution; for moderate/large n, t≈z so normal-based CIs are common.- Applied scientists rely on CLT to justify Gaussian approximations for hypothesis tests and CIs.**Why n ≈ 30 is often used**- Empirically many distributions (moderate skew/kurtosis) produce sampling distributions close to normal by n≈30.- Rule-of-thumb: 30 balances practicality and convergence speed; t and normal quantiles are close by then.**Assumptions & caveats**- Observations approximately independent and identically distributed.- Finite variance (heavy tails break CLT applicability).- No extreme skew or outliers; convergence is slower for skewed/heavy-tailed distributions.**Counterexample where n=30 may fail**- Sampling from a heavy-tailed distribution (e.g., Cauchy: no finite mean/variance) — CLT doesn’t apply; sample mean is unstable.- Highly skewed distributions (e.g., exponential with small n=30): coverage of normal CIs can be poor; bootstrap or larger n recommended.**Practical advice**- Check skewness/kurtosis, visualize bootstrap CIs, use robust estimators or increase n when tails/skewness are strong.
MediumTechnical
51 practiced
As an applied scientist, explain when to prefer cross-validation, holdout validation, or information criteria (AIC/BIC) for model selection. In particular, contrast goals of prediction versus causal inference and provide guidance for selecting models and reporting selection uncertainty.
Sample Answer
**Brief framing — prediction vs causal inference**- Prediction: goal is lowest generalization error on new data. Correlations and flexible models are fine.- Causal inference: goal is unbiased estimate of treatment effect or structural parameter; model choice emphasizes identification, confounder control, and robustness, not purely predictive fit.**When to use each method**- Cross-validation (CV) - Use for predictive modeling and hyperparameter tuning (k-fold, repeated CV). Estimates out-of-sample error and guards against overfitting across model classes (trees, regularized regressions, ensembles). - Prefer when data quantity is moderate-to-large and i.i.d. assumptions hold or when temporal CV variants (time-series split) are available.- Holdout validation - Simple, fast. Use as final sanity check when you have a large independent test set (e.g., production stream or reserved cohort). - Prefer when you need an unbiased estimate of real-world performance and can afford to set aside data.- Information criteria (AIC/BIC) - Use when models are nested or likelihood-based and emphasis is on model parsimony under parametric assumptions. - AIC targets predictive accuracy (asymptotically minimizes KL divergence); BIC favors simpler models when true model in candidate set (consistency). - Less reliable for complex, nonparametric models or when model assumptions are violated.**Causal considerations**- Avoid selecting causal adjustment sets by prediction criteria alone. Use domain knowledge and causal graphs (DAGs) to choose covariates. Use sensitivity analysis, placebo tests, and double-robust estimators to assess selection impact.**Reporting selection uncertainty**- Report CV error distributions, confidence intervals, and model comparison p-values (e.g., paired t-test or bootstrap). For likelihood methods, report ΔAIC/ΔBIC and Akaike weights.- For causal estimates, report robustness checks: alternate adjustment sets, balance diagnostics, and sensitivity bounds.- When presenting final model, include selection procedure, tuning details, and expected variability of performance/estimates.**Practical rule of thumb**- Prediction → prefer CV + large holdout for final test.- Parsimony/parametric inference → consider AIC/BIC alongside CV.- Causal inference → rely on identification strategy and robustness, not just selection metrics.
Unlock Full Question Bank
Get access to hundreds of Probability and Statistical Inference interview questions and detailed answers.
