Forecasting and Time-Series Analysis Questions
Analyzing and projecting data that moves over time. Covers trend and seasonality decomposition, forecasting approaches, demand modeling, and anomaly detection on time series. Emphasizes reasoning about baselines, drivers, and forecast reliability.
Describe the differences between top-down and bottom-up demand forecasting. For a new SKU launch in a new city, which approach would you initially prefer, and why? Include recommended data sources, advantages, disadvantages, and how you would combine both approaches over time.
Sample Answer
Direct answer
Top-down demand forecasting starts from an aggregate (category, market) estimate and disaggregates down; bottom-up starts from granular, local signal and aggregates up. For a new SKU launch in a new city - where there's no local history at all - top-down (grounded in category/market-level benchmarks and comparable-launch analogues) is the more defensible starting point, with bottom-up signal blended in as it accumulates.
Structured elaboration
- Top-down for a genuinely new SKU/new city: use category-level demand in that city, or the launch trajectory of comparable products in comparable cities, to derive an initial estimate, then allocate it down to the specific SKU using assumed or category-typical market-share proportions. Data sources: category sales in the target city, comparable-SKU launch curves elsewhere, market-sizing estimates (population, comparable-market penetration rates).
- Bottom-up for the same launch: would require building the estimate FROM the new SKU's own local sales signal - which doesn't exist yet at launch, making pure bottom-up untenable in the first weeks. Bottom-up becomes viable, and increasingly reliable, only once actual local sales data starts accumulating.
- Advantages/disadvantages: top-down is available on day one and grounded in more stable aggregate patterns, but its accuracy depends entirely on how well the chosen category/analogue benchmarks actually represent this specific SKU-city combination - a bad comparable set produces a confidently wrong number. Bottom-up is more locally accurate once it has enough of its own data, but is unusable (or dangerously noisy) at day one.
- Combining both over time: the standard approach is a smooth handoff - weight the forecast heavily toward the top-down/analogue estimate in the earliest weeks (exactly the shrinkage/pooling logic used for any short-history series), and progressively shift weight toward the SKU's own emerging bottom-up signal as more of its own history accumulates and that signal becomes more informative than the top-down prior.
- The comparable trade-off in a B2B/pipeline context: an analogous top-down-vs-bottom-up choice shows up in sales forecasting as historical trending (extrapolating past aggregate revenue, a top-down-style approach) versus pipeline-based, deal-stage forecasting (aggregating individual deal probabilities, a bottom-up-style approach) - historical trending is more stable and available even without granular pipeline data, while pipeline-based forecasting is more responsive to what's ACTUALLY currently in motion but depends on deal-stage probabilities being well-calibrated, which they often aren't early in a sales process; the same "blend, and shift weight as better signal accumulates" logic applies there too.
Worked example
For a new-city product launch, a purely top-down estimate might allocate 2% of the city's total category spend to the new SKU based on the launching brand's typical market share elsewhere; three weeks after launch, once actual local sell-through data exists, the forecast would begin blending toward a locally-calibrated estimate, fully deferring to it once enough weeks of local signal have accumulated to make the analogue-based prior no longer the more informative input.
Trade-offs & pitfalls
The most common failure is treating the top-down/analogue-based launch estimate as fixed and forgetting to actively transition toward bottom-up signal as it arrives - a forecast that never updates away from its initial top-down anchor, even after weeks of contradicting local data, is a sign the blending mechanism (not just the initial estimate) needs attention.
For store-level KPIs (sales, foot-traffic, promotions) consider multivariate methods: explain when to use Vector Autoregression (VAR), state-space (Kalman) models, or a multivariate LSTM for forecasting and causal analysis. Describe model identification, stationarity checks, stability conditions, impulse response interpretation for VAR, and validation strategies for multi-step forecasts.
Sample Answer
Direct answer
For multivariate store-level KPIs, VAR (Vector Autoregression) models several series jointly as each depending on lagged values of ALL the series, useful for both forecasting and studying how a shock to one variable propagates to the others; state-space/Kalman-filter models offer a more flexible, updatable framework for the same joint-dynamics idea with a natural way to handle noisy/partial observations and online updating; a multivariate LSTM offers the most flexible, least assumption-laden option at the cost of interpretability - the choice depends on whether you need VAR's interpretable causal-style structure, Kalman's online-updating capability, or LSTM's raw flexibility.
Structured elaboration
- VAR: each variable (sales, foot-traffic, promotions) is modeled as a linear function of lagged values of itself AND every other variable in the system; requires all variables to be stationary (checked via ADF/KPSS the same way as a univariate series, applied to each variable) and lag-order selection (via AIC/BIC over a small candidate range) before fitting.
- Model identification and stability conditions for VAR: after fitting, check the model's stability (all eigenvalues of the coefficient structure inside the unit circle) - an unstable VAR produces explosive, non-sensical long-horizon forecasts, a direct analog to a non-stationary series' unbounded extrapolation risk.
- Impulse response interpretation: VAR's signature analytical tool - trace out how a one-time shock to one variable (e.g. a promotion) propagates through the system to affect the OTHER variables (sales, foot-traffic) over subsequent periods, which is directly useful for both forecasting AND a lightweight causal-style analysis of how the variables interact, though (as with Granger causality) this reflects PREDICTIVE/statistical relationships within the modeled system, not a guaranteed causal mechanism without further justification.
- State-space (Kalman) models: represent the system's true underlying state as evolving according to a (possibly noisy, possibly time-varying) process, with the observed KPIs as noisy MEASUREMENTS of that underlying state - naturally handles missing or noisy observations, and supports genuine ONLINE (recursive, one-observation-at-a-time) updating as new data arrives, without needing to refit the whole model from scratch, which VAR's standard batch-fitting approach does not offer as naturally.
- Multivariate LSTM: makes no linear or explicit-state-space assumption at all, and can represent more complex, nonlinear interactions between the variables than either VAR or a standard Kalman model - the trade-off is needing substantially more data to fit reliably, and losing VAR's direct impulse-response interpretability and Kalman's principled online-updating framework, unless specifically engineered to approximate either.
- Validation strategies for multi-step forecasts: for any of the three, validate with a genuine multi-step-ahead backtest (not just one-step-ahead accuracy, which can look deceptively good even for a model that compounds error badly over a longer horizon) using rolling-origin evaluation, checking accuracy separately at each horizon step to see whether errors compound differently across the three model families.
Worked example
For store-level sales, foot-traffic, and promotions: a VAR fit and checked for stability would let you trace an impulse response showing how a promotion shock propagates to foot-traffic (typically a fast, short-lived bump) and then to sales (potentially a longer-lasting effect) - directly useful for planning FUTURE promotions' expected multi-variable impact, something a univariate sales-only model has no way to represent, since it can't see the promotion's effect flowing through the intermediate foot-traffic variable at all.
Trade-offs & pitfalls
The most common mistake with VAR specifically is skipping the stability check after fitting - an unstable fitted VAR can still produce plausible-looking short-horizon forecasts while being fundamentally explosive at longer horizons, a failure mode that's easy to miss without explicitly checking the eigenvalue condition rather than just eyeballing near-term forecast plots.
You observed a sudden 10% drop in weekly active users. Design a statistical test or analytic approach to decide whether this drop is due to seasonality/expected variance or a causal change from a recent deployment. Describe data selection, candidate models (seasonal decomposition, SARIMA, BSTS), use of control series, hypothesis testing, and how you'd quantify confidence in attribution.
Sample Answer
Direct answer
To decide whether a sudden drop is expected seasonal/random variance or a real causal effect from a recent change, build an explicit statistical comparison: model what the series was EXPECTED to do this period (via seasonal decomposition, SARIMA, or a Bayesian structural time series model), quantify how far the observed drop is from that expectation in probabilistic terms, and corroborate with a control series unaffected by the change wherever one is available.
Structured elaboration
- Data selection: use enough history to estimate the seasonal pattern reliably (at least a few full seasonal cycles), and be careful to exclude any period that was itself anomalous (a past outage, a past unrelated shift) from the baseline used to estimate "expected" behavior.
- Candidate models for expected behavior: seasonal decomposition (STL or classical) gives a quick expected value plus an implicit residual-based sense of normal variance; SARIMA gives a formal predictive distribution with a prediction interval; Bayesian Structural Time Series (BSTS) is specifically well-suited here because it's designed for causal-impact style analysis - it produces a full counterfactual prediction ("what would the series likely have done without the change") with a credible interval, which is a more direct answer to "is this drop unusual" than a plain decomposition.
- Using a control series: if a comparable, unaffected series exists (an unaffected region, a cohort not exposed to the deployment), compare its behavior over the same window - if the control ALSO shows a comparable drop, that's strong evidence the true cause is something shared (broader seasonality, a macro event) rather than the specific change being investigated.
- Hypothesis testing and quantifying confidence: frame it explicitly as a hypothesis test - H0: the observed value is consistent with the model's predictive distribution (i.e. explainable by normal variance); if the observed drop falls well outside the model's prediction interval (or, in a BSTS framing, the counterfactual credible interval), reject H0 in favor of a real causal effect, and report the width of that interval so stakeholders understand HOW confident the conclusion is, not just the binary verdict.
- Multiple-cohort correction: if you're checking several cohorts or segments simultaneously for the same kind of drop, correct for multiple comparisons (e.g. a Bonferroni or FDR adjustment on the p-values) - checking 20 cohorts at the standard 5% significance threshold will produce roughly one false "significant" finding by chance alone if left uncorrected.
Worked example
A first practical filter before any formal modeling: distinguish a SUSTAINED trend from a TRANSIENT anomaly with a few concrete checks - does the metric recover within a day or two (favors transient/anomaly) or does the new level persist across multiple full seasonal cycles (favors a real, sustained shift)? Is the drop isolated to one segment (a specific platform, a specific region) consistent with a localized deployment, or does it appear everywhere (favors a shared, external cause like a broad seasonal effect)? Only once these quick checks are ambiguous does the fuller BSTS/control-series analysis earn its cost.
Trade-offs & pitfalls
The most common mistake is skipping straight to "is this significant" without first checking whether the model used to define "expected" was itself well-calibrated (a seasonal model fit on a short or contaminated history will produce an overconfident interval, making ordinary variance look like a dramatic anomaly). Equally common: attributing a drop to the most recent visible change (a deployment) purely because of timing, without checking a control series - correlation in timing alone is weak evidence, and a genuinely rigorous answer needs either a true experiment (if one exists) or a credible quasi-experimental comparison.
Describe methods to detect forecast bias over time for monthly forecasts. Include which metrics to compute, statistical tests or tracking signals you would use, how to set thresholds for action, and what corrective measures you would propose once bias is confirmed. Give specific calculations you would report to stakeholders.
Sample Answer
Direct answer
Detecting forecast bias means checking whether forecast errors are systematically one-directional over time (not just noisy around zero), using a mean-percentage-error tracking signal and a formal significance test, setting an action threshold, and applying a correction (recalibration, an explicit bias term, or retraining) once bias is confirmed.
Structured elaboration and worked example (executed)
Against a synthetic 24-month series with a genuine, injected +6% systematic under-forecast bias:
pct_error = (y_true - y_pred) / y_true * 100 # positive = under-forecast
t_stat, p_value = stats.ttest_1samp(pct_error, popmean=0)
Executed result:
mean percentage error (bias): 6.33% (true injected bias: 6.0%)
one-sample t-test vs 0 (H0: no systematic bias): t=49.452, p=0.00000
-> statistically significant bias
The measured bias (6.33%) closely recovers the true injected 6%, and the one-sample t-test against zero decisively rejects "no bias" (p far below any reasonable threshold) - this is the core statistical mechanic: don't just eyeball a mean-error number, test whether it's distinguishable from zero given the observed variance, since a small sample or high-variance series can produce a nonzero-looking average purely by chance.
- Metrics to compute: mean error (in native units) and mean PERCENTAGE error (signed, not absolute - absolute error can't reveal a systematic DIRECTION, only overall magnitude) are the two core bias-tracking signals; track both over a rolling window, not just all-time, since bias can emerge or resolve over time.
- Statistical tests/tracking signals: a one-sample t-test against zero (as above) formalizes "is this bias distinguishable from noise"; a tracking signal (cumulative sum of forecast errors divided by mean absolute deviation) is a classical alternative that flags a running total drifting persistently away from zero, functionally similar in spirit to CUSUM.
- Setting thresholds for action: define both a STATISTICAL threshold (p < 0.05, or your chosen significance level) and a PRACTICAL threshold (bias small enough in absolute/business terms not to matter even if technically significant) - with enough data, even a trivially small bias can become statistically significant, so both checks matter, not just the p-value.
- Corrective measures: recalibration (apply a simple multiplicative or additive correction factor once bias is confirmed stable), an explicit bias term added to a model that structurally can't otherwise represent it, or full retraining if the bias reflects a genuine change in the underlying process rather than a fixable calibration offset.
- SQL per-SKU computation: the same logic (mean error, mean percentage error) computes directly in SQL with
AVG(actual - forecast)andAVG((actual - forecast) / actual)grouped by SKU; a real implementation trap surfaced by testing this exact query: filtering NULL actuals in the WHERE clause BEFORE grouping silently prevents you from also counting how many rows were excluded in the same query (the excluded rows are gone before the aggregate ever sees them) - the fix is to move the NULL-exclusion into the aggregate itself (AVG(CASE WHEN actual IS NOT NULL THEN ... END)alongside a separateSUM(CASE WHEN actual IS NULL THEN 1 ELSE 0 END)), which correctly reports both the bias AND the exclusion count from one query. Running the WHERE-based version against a table with one deliberately NULL row reports 0 excluded; the CASE-based fix correctly reports 1.
Trade-offs & pitfalls
Bias by region or product line should be checked SEPARATELY, not just in aggregate - a portfolio that's unbiased on average can hide an offsetting +10%/-10% bias across two segments, which is invisible to an aggregate check but very real operationally for each segment. And a corrective bias term added once should be revisited periodically, not applied and forgotten: if the underlying cause resolves (e.g. a one-off supply issue is fixed), a stale correction factor becomes a NEW source of bias in the opposite direction.
Define Exponentially Weighted Moving Average (EWMA) and provide the recursive update formula. Explain how the smoothing factor alpha controls responsiveness versus noise reduction and give an example heuristic for choosing alpha when you want to detect a sudden 10% drop in conversion rate.
Sample Answer
Direct answer
EWMA maintains a smoothed value updated recursively as St=αxt+(1−α)St−1, where alpha controls the trade-off between responsiveness (high alpha, reacts fast, passes through more noise) and noise reduction (low alpha, smoother, slower to react); a practical heuristic for choosing alpha to detect a specific-size shift (e.g. a sudden 10% drop) is to pick alpha large enough that the EWMA visibly moves toward the new level within a small number of periods, without being so large that ordinary day-to-day noise alone crosses your alerting threshold.
Structured elaboration and worked example (executed)
import numpy as np
def ewma(values, alpha):
out = np.empty_like(values, dtype=float)
out[0] = values[0]
for t in range(1, len(values)):
out[t] = alpha * values[t] + (1 - alpha) * out[t - 1]
return out
Run with alpha=0.2 on a series with a baseline around 100 that shifts to a sustained ~112 (a +12% shift) partway through:
baseline EWMA mean (last 5 of baseline): [100.88 101.52 100.32 101.03 99.65]
EWMA after shift (last 5): [110.29 109.89 112.19 110.97 110.1]
The EWMA converges close to the new level within roughly 10-15 periods at alpha=0.2. A larger alpha would converge faster but track single-point noise more closely; the choice is a genuine sensitivity/false-alarm trade-off, not something with one universally-correct answer.
- How alpha controls the trade-off: a HIGH alpha weights the newest observation heavily, so the EWMA follows the raw series closely (fast to detect a real shift, but also reacts to a single noisy point - more false alarms if used directly as an alerting signal); a LOW alpha smooths heavily over many past observations (slow to detect a real, sustained shift, but robust to single-point noise).
- Heuristic for choosing alpha to detect a specific target shift: think in terms of an effective "memory" window - an EWMA with parameter alpha weights recent observations roughly like a moving average of window size ≈α2−1; to detect a sustained 10% drop reliably within, say, 5 periods without over-reacting to single-point noise of a known typical size, choose alpha so this effective window is short enough to react within your desired detection latency, then validate empirically (as done above) that ordinary noise alone doesn't cross your intended alerting threshold at that alpha.
Trade-offs & pitfalls
EWMA (like CUSUM) is fundamentally a SMOOTHING/tracking statistic, not by itself a full alerting system - turning it into one requires an explicit threshold and a decision about how to define "baseline" in the first place (especially on a series with its own trend/seasonality, where the EWMA needs to track a MOVING baseline, not a fixed constant, or every seasonal peak will look like a shift). The single most common mistake is choosing alpha (or a threshold) by convention rather than validating against the ACTUAL noise characteristics of the specific metric you're monitoring, since two metrics with very different natural variance need different alpha/threshold combinations to hit the same false-alarm rate.
Unlock Full Question Bank
Get access to all Forecasting and Time-Series Analysis interview questions and detailed answers.
Sign in to ContinueJoin thousands of developers preparing for their dream job.