Forecasting and Time-Series Analysis Questions

Direct answer
Moving beyond raw accuracy, a forecast's business value can be measured through KPIs like stockouts prevented, holding-cost reduction, and revenue uplift - each computed by comparing what actually happened (with the forecast-driven decision in place) against a credible counterfactual of what would have happened under a naive or prior approach, with attribution challenges handled by isolating the forecast's specific contribution from other simultaneous operational changes.

Structured elaboration

• Stockouts prevented: compare the actual stockout rate under the current (forecast-informed) inventory policy against a counterfactual - either the stockout rate the OLD policy would have produced on the same demand realization (if you can simulate it), or a controlled holdout where a subset of SKUs/stores continued using the old approach as a genuine comparison group.
• Holding-cost reduction: compute the excess inventory carried under the old policy versus the new forecast-informed one, valued at the business's actual holding-cost rate (capital cost, storage, spoilage/obsolescence risk) - this requires the SAME kind of counterfactual comparison as stockouts, not just "inventory went down," since inventory could fall for unrelated reasons (a demand drop, say).
• Revenue uplift: the incremental revenue attributable to better availability/staffing/allocation decisions the improved forecast enabled, computed similarly by comparison against a credible counterfactual rather than simply period-over-period revenue growth (which conflates many causes).
• Computing these from historical forecasts and actuals: requires not just the forecast and the actual outcome, but a record of what DECISION the forecast drove (the resulting inventory order, staffing level, allocation) so you can quantify the downstream operational consequence, not just the raw forecast error - a forecast can be numerically accurate yet drive a poor decision if the decision rule built on top of it (e.g. the safety-stock formula) is miscalibrated.
• Attribution challenges when multiple changes happen simultaneously: if inventory policy, pricing, AND the forecasting model all changed in the same quarter, isolating the forecast's specific contribution to any observed improvement requires either staggering the rollouts (so each change's effect window is at least partially separable), a genuine holdout/control group that kept the old forecast while everything else changed identically, or, at minimum, an honest acknowledgment in the reporting that the estimate is a joint effect of several changes rather than the forecast's alone.

Worked example
A holding-cost-reduction claim of "$500K saved this quarter" is only credible if it's benchmarked against a stated counterfactual (e.g. "versus the safety-stock levels the prior policy would have required for the same realized demand") - reported without that counterfactual, the number is really just "inventory happened to be lower," which could equally reflect a demand slowdown having nothing to do with forecast quality.

Trade-offs & pitfalls
The recurring failure mode across all three KPIs is presenting a raw before/after comparison as if it were the forecast's isolated causal contribution, when in a real operating business, demand, pricing, promotions, and policy all move simultaneously - treat any business-value number without an explicit counterfactual or control group as a directional estimate, not a precise measurement, and say so plainly when reporting it.

Direct answer
ARIMA(p,d,q) models a series as a combination of three pieces: AR(p), autoregression on the series' own past p values; I(d), the number of times you difference the series to make it stationary; and MA(q), a moving average of the past q forecast errors. You identify p and q by reading the ACF and PACF plots of the (differenced) series, and you add seasonal terms (SARIMA) when the ACF/PACF still show a repeating spike pattern at the seasonal lag after ordinary differencing.

Structured elaboration

• AR(p): today's value is a linear function of the last p values plus noise. A PACF that cuts off sharply after lag p (with earlier lags significant) points to an AR(p) term, because the PACF isolates the direct effect of each lag after removing the effect of the lags in between.
• I(d): the number of times you difference $y_t$ (i.e. work with $y_t - y_{t-1}$, or the second difference) before the series is stationary. You determine d with a stationarity test (ADF/KPSS) rather than by eye: difference until the test says stationary, and stop as soon as it does. Over-differencing (d too large) introduces artificial negative autocorrelation and inflates forecast variance.
• MA(q): today's value depends on the last q forecast errors, not raw values. An ACF that cuts off sharply after lag q (while the PACF decays slowly) points to an MA(q) term.
• Order identification in practice: plot ACF and PACF of the differenced series. AR signature = PACF cuts off, ACF tails off. MA signature = ACF cuts off, PACF tails off. Mixed ARMA signatures (both tail off) are common in practice, so analysts usually also compare a small grid of candidate (p,d,q) by AIC/BIC rather than trusting the plots alone.
• When to add seasonal terms (SARIMA): if, after taking the ordinary difference, the ACF/PACF still show significant spikes at the seasonal period (e.g. lag 7 for daily-with-weekly-seasonality data, lag 12 for monthly-with-yearly-seasonality), a plain ARIMA hasn't captured the seasonal structure. SARIMA adds seasonal AR/MA/differencing terms $(P,D,Q)_s$ operating at multiples of the seasonal period s, on top of the ordinary $(p,d,q)$ terms.

Worked example
Take daily retail sales with a weekly pattern. Fitting an ARIMA(1,1,1) leaves a residual ACF with a clear spike at lag 7 (and 14, 21) - the model has removed the trend (via d=1) but not the weekly repetition. Adding a seasonal term, e.g. SARIMA(1,1,1)(1,0,1)$_7$, lets the seasonal AR/MA terms absorb that lag-7 structure; after refitting, the residual ACF should show no significant spikes at multiples of 7. Concretely: raising d from 0 to 1 changes what "capturing p and q" even means, since AR(p)/MA(q) are now fit on the differenced series, not the raw one - a common early mistake is reading ACF/PACF on the raw series and getting orders that don't apply once differencing is applied.

Trade-offs & pitfalls
Adding seasonal terms multiplies the number of parameters and the search space (p,d,q,P,D,Q,s); over-specifying any one of them (especially D, seasonal differencing) can remove real signal along with the seasonality. A senior candidate will also flag that pure ACF/PACF reading is a starting point, not a final answer: automated order search guided by AIC/BIC (or pmdarima.auto_arima) is standard practice once the visual signature is ambiguous, and the final choice should always be checked with a residual diagnostic (no significant autocorrelation left, e.g. a Ljung-Box test) rather than trusted from the identification step alone.

Direct answer
Forecasting a demand or capacity metric (like viewing hours, order volume, or active users) with strong seasonality and event-driven spikes calls for the same core toolkit as any seasonal demand forecast: a classical or ML model chosen to fit the seasonality's complexity, explicit handling of known future events (releases, holidays, campaigns), and evaluation against the operational decision the forecast actually supports (a capacity or staffing plan), not just a generic accuracy number.

Structured elaboration

• Model choice, classical vs ML: for one series with clean weekly+monthly seasonality and no meaningful external drivers, SARIMA or Holt-Winters are strong starting points. Once you need to incorporate known upcoming events (a title release, a new-market launch, a promotional campaign) as explicit regressors, or you're forecasting many related series together (many titles, many regions, many order channels), a model that supports exogenous features and pooling - Prophet with regressors, or a gradient-boosted model over lag/calendar/event features trained across all series - is the better fit.
• Handling known future events: build an explicit event/release-calendar feature (a flag, or a decaying "days since release" feature capturing the post-spike decline) rather than hoping the model infers a one-off spike from history alone; a genuinely new event type (first-ever release in a new content category) has no historical analogue for the model to learn from, so pair the model output with a judgmental adjustment or an analogue-based estimate for events without a clean precedent.
• Evaluating forecast accuracy for planning purposes: report error metrics AT THE GRANULARITY the planning decision needs (e.g. hourly RMSE if staffing is scheduled hourly, not just an aggregated daily/weekly number that could hide a systematically-missed daily peak), and evaluate coverage of the prediction interval specifically around event days, since those are exactly when planners most need the interval to be trustworthy and are also where models are most likely to be poorly calibrated.
• Scaling considerations when the metric feeds a real capacity decision: because under-forecasting a spike has an asymmetric cost (a capacity shortfall vs a modest over-provisioning cost), it's often appropriate to plan off a higher quantile of the forecast distribution (e.g. the 90th percentile) rather than the point median, and to build in an explicit contingency/fallback plan (manual override, rapid-scale trigger) for events materially larger than anything in the training history.

Worked example
Growth-style capacity questions (e.g. planning for user growth from 100k to 2M over 12 months) are a related but DISTINCT variant: at that scale of change, a pure time-series extrapolation of recent history is unreliable because the growth is driven by a step-change in the business (a new market, a new channel), not organic continuation of an existing trend, so a growth-curve or scenario-based model (grounded in comparable prior launches, if any exist) is usually more defensible than SARIMA/ETS extrapolation alone, with prediction intervals presented explicitly as scenario bands rather than a single confident number. A supply-side shock (e.g. a courier strike suddenly reducing available capacity) is the mirror-image problem: the DEMAND forecast may still be accurate, but the operational decision needs a separate supply-constraint model layered on top, since the two are not interchangeable.

Trade-offs & pitfalls
The most common mistake is validating the forecast on average-case error while the capacity decision actually depends on TAIL behavior (event days, growth inflection points, supply shocks); a model that looks excellent on an aggregate accuracy metric can still be dangerously wrong exactly when the capacity plan needs it most. Report event-day accuracy and interval coverage separately from the steady-state number, and be explicit with stakeholders about which regime (steady-state vs event-driven vs step-change growth) a given forecast is actually reliable for.

Direct answer
Responding to a sudden, dramatic demand shock (like a 70% drop with a changed seasonal pattern) means moving fast through rapid diagnostics, short-term operational fallbacks that don't depend on a not-yet-retrained model, a deliberate retraining strategy once the new regime is better understood, explicit scenario planning for recovery, and continuous, honest uncertainty communication throughout - in roughly that priority order.

Structured elaboration

• Rapid diagnostics to quantify the change: first establish the basic facts fast - how large is the drop, is it uniform across segments/regions or concentrated, and has the WEEKLY seasonal pattern itself genuinely changed shape (not just shifted level) - this initial read determines how radically the forecasting approach needs to change, versus a milder response being sufficient.
• Short-term heuristic fallbacks for operations: a model trained entirely on pre-shock history will actively mislead if used naively during the shock itself (extrapolating a pattern that no longer holds); a simple, transparent heuristic (e.g. "current week's actuals, or a short recent-window average, rather than the stale model's output") is often a more honest and more useful stopgap than a sophisticated model confidently extrapolating an outdated pattern.
• Retraining strategy: once enough post-shock data has accumulated to characterize the NEW pattern (not just the initial shock transient), retrain - but be deliberate about the training window: including too much pre-shock history dilutes the new pattern, too little post-shock history produces an unstable, premature re-estimate; a staged approach (retrain on a growing post-shock window as it becomes available, rather than waiting for a fixed amount before doing anything) balances this.
• Scenario planning for recovery paths: since the actual recovery trajectory (fast rebound, slow recovery, a new permanent lower baseline, a different shape than pre-shock) is genuinely uncertain during the shock itself, present multiple named scenarios rather than a single point forecast - this is exactly the same "acknowledge genuine deep uncertainty explicitly, rather than false point-forecast precision" discipline used for any severely disrupted or unprecedented forecasting situation.
• Adapting dashboards: any dashboard panel built on WoW/YoY percentage-change comparisons, or on alert thresholds calibrated against the pre-shock baseline, will either look absurdly extreme (a 70% YoY drop swamping every other number on the page) or silently mislead (thresholds firing constantly, or not firing at all, against a baseline that's no longer valid). Annotate the shock period explicitly on any historical-trend chart so viewers don't misread the discontinuity as a data error or pipeline bug; temporarily suspend or clearly flag any YoY/WoW comparison metric whose window spans the shock boundary, since the underlying comparison is no longer meaningful; and recalibrate alert thresholds against the NEW short-term-heuristic baseline rather than the stale pre-shock one, revisiting both the annotations and the thresholds again once retraining and scenario tracking are underway.
• Communicating uncertainty and recommended actions: to operations, translate directly into concrete near-term actions under the current best-guess scenario (staffing, inventory); to executives, communicate the RANGE of plausible recovery scenarios and what data signal would indicate which one is actually materializing, framed as "here's what we're watching to tell these scenarios apart" rather than false confidence in one specific path.

Worked example
In the first days of such a shock: report the magnitude and shape of the drop (rapid diagnostics), recommend operations use a short recent-window heuristic rather than the stale model's now-invalid output (short-term fallback), and present 2-3 named recovery scenarios (e.g. "sharp V-shaped rebound within weeks," "gradual multi-month recovery," "a new, lower permanent baseline") with the specific signals (weekly growth rate, whether the pre-shock weekly seasonal shape has resumed) that would distinguish which scenario is actually unfolding as more data arrives, updating the recommendation as that evidence accumulates rather than committing to one scenario prematurely.

Trade-offs & pitfalls
The most damaging mistake in a disruption like this is continuing to trust (or silently keep serving) a model's output as if nothing had changed, simply because retraining hasn't happened yet - an explicit, fast decision to fall back to a transparent heuristic during the gap between "the old model is now known to be wrong" and "a properly retrained model is ready" is what protects operational decisions during exactly the period they're most vulnerable.

Direct answer
Resolving a cross-functional disagreement about retrain frequency and prediction TTL means making the trade-off (accuracy vs compute cost vs operational stability) explicit and quantified rather than argued in the abstract, aligning stakeholders around a shared decision framework rather than a single number, and building in a defined process to revisit the decision as conditions change rather than treating it as permanently settled.

Structured elaboration

• Evaluating the trade-off: quantify how much forecast accuracy actually degrades as prediction TTL (time-to-live, how long a forecast is served before being refreshed) increases - this is directly measurable via backtesting (compare accuracy of a forecast used immediately vs one used N days stale); quantify the COMPUTE cost of more frequent retraining (a real, budgetable number); and separately name the OPERATIONAL STABILITY cost of frequent changes (a forecast that changes too often can itself be disruptive to downstream planning processes that expect some consistency, distinct from either accuracy or raw compute cost).
• Aligning stakeholders: bring Product, Ops, and Data Engineering into a SHARED view of the actual trade-off curve (accuracy degradation vs TTL, compute cost vs retrain frequency) rather than each side arguing from its own priority in isolation - a concrete, data-grounded trade-off curve turns an abstract disagreement ("we need fresher forecasts" vs "retraining is expensive") into a specific, negotiable point on a shared curve.
• Making and communicating a data-driven recommendation: propose a specific retrain cadence and TTL grounded in where the accuracy-degradation curve starts to bend meaningfully (the point past which staleness costs materially more accuracy than it saves in compute/stability), explicitly stating the trade-offs being accepted at that point, so the decision is legible and defensible to all three stakeholder groups rather than feeling arbitrary to whichever side "lost."
• A process for revisiting the decision: define explicit triggers for reconsidering the choice (a monitored accuracy-degradation signal crossing a threshold, a meaningful compute-cost change, a business requirement shift) rather than either revisiting it constantly (churn, instability) or never revisiting it (staleness accumulating silently) - this closes the loop the same way ongoing calibration/drift monitoring does for a model's outputs, applied here to an operational POLICY decision rather than the model itself.

Worked example
A concrete quantified trade-off might show that forecast accuracy degrades only mildly from a 1-day to a 3-day TTL, but meaningfully more from 3 to 7 days, while retrain compute cost scales roughly linearly with frequency - this evidence supports a recommendation of a 3-day TTL as the point where you're not leaving much accuracy on the table relative to daily retraining, while meaningfully reducing compute cost and operational churn relative to it, a specific, defensible number rather than either extreme position (daily retraining "because fresher is always better," or infrequent retraining "because compute is expensive") argued from principle alone.

Trade-offs & pitfalls
The most common failure in this kind of cross-functional disagreement is treating it as a one-time negotiation to "win," rather than establishing a durable, revisitable process - a decision made once and never revisited will eventually become wrong as the underlying trade-off curve itself shifts (compute costs change, the business's tolerance for staleness changes, the model's own accuracy characteristics change), and a stakeholder group that felt unheard in the original decision is far more likely to escalate the disagreement again later if there's no defined mechanism for it to be legitimately reconsidered.