Data Analysis and Insight Generation Questions

Ability to convert raw data into clear, evidence based business insights and prioritized recommendations. Candidates should demonstrate end to end analytical thinking including data cleaning and validation, exploratory analysis, summary statistics, distributions, aggregations, pivot tables, time series and trend analysis, segmentation and cohort analysis, anomaly detection, and interpretation of relationships between metrics. This topic covers hypothesis generation and validation, basic statistical testing, controlled experiments and split testing, sensitivity and robustness checks, and sense checking results against domain knowledge. It emphasizes connecting metrics to business outcomes, defining success criteria and measurement plans, synthesizing quantitative and qualitative evidence, and prioritizing recommendations based on impact feasibility risk and dependencies. Practical communication skills are assessed including charting dashboards crafting concise narratives and tailoring findings to non technical and technical stakeholders, along with documenting next steps experiments and how outcomes will be measured.

HardSystem Design

67 practiced

You must forecast weekly demand for 10,000 SKUs across multiple countries at scale. Describe an architecture and modeling strategy that balances accuracy and operational complexity: approaches for per-SKU models vs pooled/hierarchical models, feature engineering (holidays, price, promotions), model choices (Prophet, ETS, gradient boosting, RNNs), and how to monitor model performance in production.

Sample Answer

Requirements & constraints:- Forecast weekly demand for ~10k SKUs across multiple countries, with mixed history lengths, cold-start SKUs, promotions, calendar events; operational needs: daily retrain/weekly forecast, low-latency scoring, explainability, monitoring, and automated fallback.

High-level architecture:- Ingest layer: ETL pipelines (Airflow/Prefect) gather sales, price, promotions, inventory, catalog, holidays, macro signals into a centralized time-series store (Parquet on S3 / Delta Lake) and a feature store (Feast).- Training layer: Batch/stream preprocessing jobs create aggregated weekly series and feature matrices; store metadata (hierarchy, country, lifecycle).- Modeling layer: Orchestrated training with model registry (MLflow), serving via batch jobs or REST endpoints.- Monitoring: Metrics pipeline (Prometheus + Grafana) and data drift/performance alerts, automated fallback to simpler models.

Modeling strategy — balancing accuracy vs ops complexity:- Hybrid approach: - Pooled/hierarchical models as default: train global gradient boosting models (LightGBM / CatBoost) using SKU and country embeddings and hierarchical reconciliation (bottom-up or MinT). Advantages: handles sparse SKUs, lower ops, fast training/serving, strong with cross-series signals. - Per-SKU or per-category models for high-value SKUs: for top N SKUs by revenue, train specialized models (ETS/Prophet or RNN/DeepAR) to capture SKU-specific seasonality/patterns. Use automated champion-challenger to compare. - Hierarchical time-series models: combine global model forecasts with statistical models (ETS/seasonal ARIMA/Prophet) at aggregated levels and reconcile with SKU forecasts to enforce consistency across countries/categories.

Feature engineering:- Calendar: country-specific holidays, school breaks, day-of-week, week-of-year, rolling seasonality indicators.- Price & promotion: current price, promo flag, feature encoding of promotion type, lagged price elasticity features, promotional cadence (count of promo weeks in last k weeks).- Demand context: on-hand inventory, lead-time, channel mix, competitor prices (if available), weather/macro indicators.- Lags & rolling stats: lagged sales (1,2,4,13 weeks), rolling means/std, momentum, time-since-last-promo.- Cold-start: use category-level aggregates, product attributes, and embeddings from metadata to bootstrap.

Model choices & when:- Gradient boosting (LightGBM/CatBoost): primary pooled model — fast, interpretable (SHAP), handles heterogeneous features, robust to missing.- ETS/Prophet: baseline and for SKU-level seasonal patterns; useful as fallback and for explainability.- Probabilistic models (DeepAR, N-BEATS): for top SKUs requiring probabilistic forecasts — capture complex/non-linear patterns, provide quantiles.- Ensembles: blend GBDT + Prophet/Deep models via stacking or weighted averages; calibrate quantiles through isotonic regression.

Operational considerations:- Training cadence: full retrain weekly; incremental updates daily for features and weights for pooled models.- Serving: batch generation of next-12-week forecasts stored in DB; on-demand endpoint for special queries.- Reconciliation: apply hierarchical reconciliation (MinT) after forecasting to ensure sum consistency.

Monitoring & lifecycle:- Performance metrics: item-level MAPE/SMAPE, weighted MAPE by revenue, pinball loss for quantiles, calibration checks.- Explainability: SHAP summaries for GBDT; per-forecast feature attributions and human-readable explanations (e.g., “promo increased demand by X%”).- Drift detection: monitor feature distributions and residuals; use PSI and KL divergence; automated alerts when error degrades beyond thresholds or PSI > threshold.- Automated actions: champion-challenger deployment, automatic rollback to simpler baseline (moving average/ETS) if new model underperforms, periodic re-evaluation of which SKUs need per-SKU models.- CI/CD & governance: unit/integration tests for data pipelines, model validation tests, model cards, and retrain triggers based on performance windows.

Trade-offs:- Global pooled models reduce ops and help cold-starts but may miss SKU nuances; per-SKU and deep models improve accuracy for high-value SKUs at higher compute and maintenance cost. Use a tiered approach: most SKUs = pooled; top SKUs = per-SKU/deep; reconcile and monitor to balance accuracy vs complexity.

EasyTechnical

53 practiced

Explain the basic idea of time series decomposition into trend, seasonality, and residual. Provide a concrete example of how you would use decomposition to inform a business decision (e.g., staffing, promotions) and mention any limitations of classical decomposition.

MediumTechnical

49 practiced

A company rolled out a new feature in Region A on March 1 and did not roll it out in Region B. Describe how you would use a difference-in-differences (DiD) approach to estimate the causal effect on conversion. Specify data requirements, the DiD regression specification, parallel trends assumption, and diagnostic checks you would run to validate the design.

Sample Answer

Goal: estimate causal effect of the feature rollout in Region A (treated) on conversion using difference-in-differences (DiD).

1) Data requirements- Panel or repeated cross-section user-level (or aggregated) data with timestamps covering a pre-period (e.g., several weeks/months before March 1) and post-period (several weeks/months after).- Treatment indicator: Region (A=1 treated, B=0 control).- Time indicator: Post = 1 for dates >= March 1, 0 otherwise.- Outcome: conversion (binary or rate), plus user covariates (age, device, acquisition channel), seasonality controls, and sample weights if needed.- Sufficient parallel pre-period observations and similar composition across regions.

2) DiD regression specification- Baseline (user-level, linear probability or aggregated with logs if rates):Conversion_it = α + β·Treated_i + γ·Post_t + δ·(Treated_i × Post_t) + X_it'θ + μ_t + ε_it- δ is the DiD estimator (causal effect). Include time fixed effects μ_t (day/week) and optionally region fixed effects (if using more than two regions). Cluster standard errors at the region level (or higher, e.g., state) — with only two clusters interpret carefully; better if multiple regions available.

3) Parallel trends assumption- Assumes treated and control would have followed same conversion trend absent treatment. This is fundamentally untestable but can be made more plausible by showing pre-treatment trends are parallel and by conditioning on covariates.

4) Diagnostic checks & robustness- Event-study (dynamic DiD): estimate leads and lags to visualize pre-trends and dynamics:Conversion_it = α + Σ_k≠-1 δ_k · (Treated_i × I[t = k]) + X_it'θ + μ_t + ε_itCheck δ_k ≈ 0 for k < 0 (no pre-trend).- Pre-trend test: regress pre-period outcomes on treatment*time interaction; test joint significance.- Placebo test: assign pseudo treatment dates before March 1 or randomize region labels to ensure no effect.- Covariate balance & time-varying covariates: check that observable covariates do not change differentially post in treated vs control; if they do, include them or use inverse probability weighting / matching on pre-period trends.- Heterogeneity: estimate effects by segment (new vs returning users, device) to check consistency.- Alternative specifications: use logistic/GLM for binary outcomes, aggregated rate models with Poisson/quasi-Poisson, or synthetic control if single treated region and many controls.- Sensitivity: test different pre/post windows; cluster-robust SEs; wild bootstrap if few clusters.

5) Interpretation & limitations- If pre-trends are flat and robustness checks pass, interpret δ as causal average treatment effect on conversions. If pre-trends fail or only two regions exist, be cautious; consider collecting more control regions or using synthetic control.

HardTechnical

55 practiced

Describe methods to handle data that is missing-not-at-random (MNAR). Provide a practical plan including sensitivity analysis, selection models (e.g., Heckman), and bounding approaches. Explain how you would present the robustness of your conclusions to stakeholders when MNAR is plausible.

Sample Answer

Start with clarifying the problem: MNAR means probability of missing depends on unobserved values (or on missingness itself). That breaks MCAR/MAR assumptions and can bias estimates; we must both model and communicate uncertainty.

Practical plan1) Exploratory diagnostics- Compare observed vs auxiliary variables, plot patterns, use logistic models of missingness on observed covariates to see predictors.- Seek external/auxiliary data (surveys, prior periods, instrument variables) that correlate with the missing value.

2) Baseline MAR analyses- Fit models under MAR (multiple imputation, ML with missingness indicators) to produce a reference estimate and to identify sensitivity to covariates.

3) Selection / shared-parameter models- Implement explicit MNAR models: Heckman-type two-step for continuous outcomes with sample selection; joint models or selection models where you model outcome and missingness mechanism simultaneously (e.g., logistic selection with outcome-dependent term). Estimate by MLE or Bayesian methods (Stan) to propagate uncertainty.

4) Pattern-mixture and sensitivity analysis- Use pattern-mixture models: impute under varying plausible shifts delta to the imputed values (delta-adjustment) and observe effect on target metrics.- Perform tipping-point analysis: find the MNAR parameter values that change a decision (e.g., significant to non-significant, or ROI threshold).- Use Bayesian priors on MNAR parameters to encode expert beliefs and produce posterior intervals.

5) Bounding approaches- Apply nonparametric bounds (Manski bounds) to get worst-and-best-case ranges when assumptions are minimal. Combine with covariate stratification to tighten bounds.

Communication to stakeholders- Show baseline MAR estimate plus a concise sensitivity band: e.g., point estimate with shaded region from plausible MNAR deltas and the full Manski bounds.- Present tipping points: “To reverse this decision effect, missing values would need to be X units lower/higher—this is or isn’t plausible based on domain knowledge.”- Offer recommendations: collect targeted follow-up data, instrument variables, or conservative policies if conclusions are sensitive.- Be explicit about assumptions, show how much conclusions depend on them, and supply code & reproducible notebooks so analysts can inspect alternatives.

Key takeaway: combine diagnostics, explicit MNAR modeling, sensitivity grids/tipping points, and bounding to quantify robustness; present results as ranges tied to clear assumptions so stakeholders can make informed, risk-aware decisions.

EasyTechnical

97 practiced

In Python (pandas), write code to: 1) remove duplicate rows based on ['user_id','occurred_at'], 2) parse occurred_at into datetime and set as index, 3) convert 'value' to numeric, 4) impute missing numeric 'value' with the median, and 5) impute missing categorical 'platform' with the most common value. Use the following sample schema for context: DataFrame columns = ['user_id', 'event_type', 'value', 'platform', 'occurred_at']. You do not need to write imports but show the core transformations in pandas.

Sample Answer

Approach: drop duplicates on ['user_id','occurred_at'], parse occurred_at to datetime and set as index, coerce 'value' to numeric, fill numeric NaNs with median, fill categorical 'platform' NaNs with mode.

python

# assume df is the input DataFrame

# 1) remove duplicates based on user_id and occurred_at (keep first)
df = df.drop_duplicates(subset=['user_id', 'occurred_at'], keep='first')

# 2) parse occurred_at to datetime and set as index
df['occurred_at'] = pd.to_datetime(df['occurred_at'], errors='coerce')  # coerces bad strings to NaT
df = df.set_index('occurred_at').sort_index()

# 3) convert 'value' to numeric (coerce non-numeric to NaN)
df['value'] = pd.to_numeric(df['value'], errors='coerce')

# 4) impute missing numeric 'value' with median
median_val = df['value'].median(skipna=True)
df['value'] = df['value'].fillna(median_val)

# 5) impute missing categorical 'platform' with the most common value (mode)
if df['platform'].dropna().empty:
    mode_platform = None
else:
    mode_platform = df['platform'].mode().iloc[0]
df['platform'] = df['platform'].fillna(mode_platform)

Key points:- errors='coerce' prevents exceptions and marks unparsable as NaT/NaN.- median is robust to outliers; mode handles categorical imputation.- After setting datetime index, time-based operations become easier.

Unlock Full Question Bank

Get access to hundreds of Data Analysis and Insight Generation interview questions and detailed answers.

Join thousands of developers preparing for their dream job.