Forecasting and Time-Series Analysis Questions

Direct answer
For heavy-tailed count data with occasional extreme peaks, a plain squared-error objective lets the rare large peaks dominate training; log-transforming the target, using a Poisson or negative-binomial objective (which natively model count data's mean-variance relationship), or using Huber loss (which caps the influence of large errors) are the standard remedies, each with different trade-offs.

Structured elaboration

• Log-transform: train on $\log(1+y)$ instead of raw $y$, which compresses the scale of extreme peaks relative to typical values before the model even sees them, effectively de-weighting their influence on a standard squared-error objective; simple to implement with any standard regression objective, but requires careful back-transformation (and correction, since a naive back-transform of a log-space prediction is biased low for the expected value on the original scale) to get predictions back onto the original count scale.
• Poisson objective: natively models count data's characteristic property that variance scales with the mean (higher-volume zones naturally have more absolute variability, not just more absolute counts) - appropriate when the data's mean-variance relationship is genuinely close to Poisson's assumption (variance ≈ mean); the model directly optimizes count-appropriate likelihood rather than a generic error function.
• Negative-binomial objective: relaxes Poisson's often-too-restrictive assumption (variance = mean exactly) to allow variance to exceed the mean (overdispersion), which is very common in real business count data (demand counts are usually MORE variable than a pure Poisson process would predict) - generally the more realistic default of the two count-specific objectives for genuinely overdispersed business data.
• Huber loss: behaves like squared error for small residuals (smooth, well-behaved gradient near zero) but switches to a LINEAR penalty beyond a chosen threshold, capping how much any single large-residual outlier can dominate the overall loss - doesn't assume anything about the target's distributional shape (unlike Poisson/NB), making it a simpler, more general-purpose robustness tool, at the cost of not exploiting the specific count-data structure the Poisson/NB objectives do.
• Trade-offs among the three: log-transform is simple and general but needs careful back-transformation and doesn't explicitly model the count structure; Poisson/NB objectives directly encode the domain-appropriate mean-variance relationship (more principled for genuine count data) but require checking which of the two (Poisson vs NB) actually fits your data's dispersion; Huber loss is agnostic to the target's distribution and simplest to reason about, but is a purely mechanical robustness fix rather than a domain-appropriate model of counts.

Worked example
For zone-level order counts where most zones see modest, fairly-regular volume but a handful of large-event zones occasionally spike to many multiples of their typical count: fitting with plain squared error would let those rare spike-zones dominate the overall loss, potentially degrading accuracy on the much more numerous, well-behaved typical zones just to marginally chase the rare spikes; switching to a negative-binomial objective (checking empirically that variance genuinely exceeds the mean across zones, which overdispersed business counts usually do) directly represents this heteroscedastic count structure without needing a separate transform-and-back-transform step.

Trade-offs & pitfalls
Always check whether your data's variance-to-mean relationship is closer to Poisson (variance ≈ mean) or meaningfully overdispersed (favoring negative-binomial) BEFORE choosing between them, rather than defaulting to one out of habit - fitting a Poisson objective to genuinely overdispersed data will produce systematically overconfident (too-narrow) implied uncertainty even if the point forecasts themselves look reasonable.

Direct answer
Transformer-based forecasting models (Temporal Fusion Transformer, Informer, Autoformer) adapt the attention mechanism, originally built for language, to time series by letting the model directly attend to any past time step when producing a forecast, capturing long-range dependencies more directly than an RNN's sequential state-passing; they also provide explicit architecture for incorporating both STATIC covariates (features that don't change over time, like a store's region) and TIME-VARYING covariates (features that change per time step, like daily promotions), at the cost of more training complexity and reduced interpretability relative to classical statistical models.

Structured elaboration

• Attention for long-range dependencies: an RNN/LSTM has to pass information through a sequential chain of hidden states, which can dilute or lose signal from far in the past by the time it reaches the current step; attention instead lets the model compute a direct, weighted connection between the current prediction and ANY past time step, regardless of distance, which is particularly valuable for time series with important long-range structure (a yearly effect in daily data, say) that a purely sequential architecture has to "remember" across many steps.
• Model-specific adaptations: the Temporal Fusion Transformer (TFT) explicitly separates static, known-future (e.g. calendar/promotion), and observed-past-only covariates into distinct input paths, with a variable-selection mechanism that learns which features matter most; Informer addresses the quadratic compute cost of standard attention (which scales poorly with long sequences) with a sparser attention mechanism designed for long-sequence forecasting; Autoformer builds decomposition (trend/seasonal, similar in spirit to classical decomposition) directly into the architecture rather than leaving the model to learn it implicitly.
• Integrating static and time-varying covariates: static covariates (store region, product category) are typically embedded once and combined with the sequence representation; time-varying covariates that are KNOWN in advance for the forecast horizon (a promotion calendar) are fed differently than ones only observed in the past (actual historical weather) - architectures like TFT make this distinction explicit in their input structure, which matters practically since it maps directly onto the "must be known at forecast time" constraint that governs any exogenous regressor.
• Trade-offs vs RNNs and classic statistical models: Transformer-based models generally need substantially more training data and compute than either RNNs or classical statistical models to realize their advantage, and are the LEAST interpretable of the three families (attention weights offer some insight but nowhere near the direct interpretability of a SARIMA coefficient or a Prophet component); their main payoff is when you have long sequences with genuine long-range dependencies AND enough data (often pooled across many related series) to train them well - on a single short series with simple seasonality, a classical model will very likely match or beat a Transformer while being far cheaper and more interpretable.

Worked example
A closely related architectural variant worth naming: a spatio-temporal Graph Neural Network (GNN) for forecasting demand across many geographically-adjacent zones, which extends the same "attend across many related sequences" idea from the temporal dimension (Transformers attending across TIME) into the spatial dimension as well (attending across geographically or logically related ZONES via an explicit adjacency/graph structure) - appropriate when neighboring zones' demand genuinely informs each other's forecast (e.g. spillover effects between adjacent delivery zones), a relationship a per-zone Transformer or RNN forecasting each zone independently has no native way to represent.

Trade-offs & pitfalls
The most common misapplication is reaching for a Transformer-based architecture on a problem that doesn't have the scale (many related series, long useful history) or genuine long-range dependency structure to justify its cost - validate the added complexity is earning its keep with a direct backtested comparison against a well-tuned classical or gradient-boosted baseline before committing to the heavier architecture and its associated training/maintenance burden.

Direct answer
Scaling forecasting across 10,000 SKUs with many short/sparse histories requires a global-vs-local model-selection strategy (most SKUs get a shared "global" model, only high-volume SKUs justify their own local model), grouping/clustering similar SKUs, a feature store serving both offline training and online inference consistently, monitoring at the segment level (not per-SKU), and explicit prioritization of which forecasts get manual review.

Structured elaboration

• Global vs local model selection: a single local (per-series) model needs enough of its OWN history to fit reliably - most of a 10,000-SKU catalog won't have that, especially new or slow-moving SKUs. A global model (one shared model trained across all SKUs, with SKU identity/features as inputs) pools information across series, which is exactly what a sparse-history SKU needs; reserve dedicated local models for the small minority of high-volume, high-value SKUs where the extra effort is justified by business impact.
• Grouping/clustering SKUs: cluster by demand pattern (volume tier, seasonality shape, category) so the global model can use cluster membership as a feature, or so you can train a handful of cluster-specific global models instead of one undifferentiated one - this captures much of the benefit of "local" specialization without needing per-SKU history.
• Feature store considerations: offline (training) and online (serving) features must be computed with the SAME logic to avoid train/serve skew - a feature store that serves both from one definition (rather than reimplementing feature logic twice) is the standard way to guarantee that. At this scale, point-in-time correctness (not leaking future data into a training row) becomes an operational requirement, not just a modeling nicety.
• Monitoring at scale: tracking accuracy per-SKU for 10,000 series is noisy and not actionable; monitor aggregated by segment/cluster (median MASE per cluster, e.g.) and surface only SKUs whose error is an outlier WITHIN their own segment for review.
• Prioritization for manual review: rank SKUs by a combination of forecast uncertainty and business impact (revenue, stockout cost) so a small analyst team's attention goes to the highest-value, least-certain forecasts rather than being spread evenly (and therefore too thin) across all 10,000.

Worked example
A retailer's catalog might cluster into "fast movers" (a few hundred SKUs with 2+ years of clean history - worth dedicated per-SKU or small-group models), "steady mid-tier" (a few thousand SKUs - served well by a single global gradient-boosted model with SKU-embedding or cluster features), and "long tail / new" (the bulk of the remaining SKUs, often with under 3 months of history - served by the same global model, leaning on pooled/cross-SKU signal since there isn't enough series-specific data to do otherwise, similar in spirit to a cold-start forecasting approach).

Trade-offs & pitfalls
Cost and compute scale with the NUMBER of models, not directly with the number of series, so a "fit everything locally" instinct becomes operationally and financially unworkable well before 10,000 SKUs; a global-model architecture is usually the right default, with local models reserved as a deliberate, justified exception. Watch specifically for a feature store skew bug (offline features computed differently than online-served features) as the failure mode that's both easiest to introduce at this scale and hardest to detect without dedicated parity monitoring.

Direct answer
ARIMA(p,d,q) models a series as a combination of three pieces: AR(p), autoregression on the series' own past p values; I(d), the number of times you difference the series to make it stationary; and MA(q), a moving average of the past q forecast errors. You identify p and q by reading the ACF and PACF plots of the (differenced) series, and you add seasonal terms (SARIMA) when the ACF/PACF still show a repeating spike pattern at the seasonal lag after ordinary differencing.

Structured elaboration

• AR(p): today's value is a linear function of the last p values plus noise. A PACF that cuts off sharply after lag p (with earlier lags significant) points to an AR(p) term, because the PACF isolates the direct effect of each lag after removing the effect of the lags in between.
• I(d): the number of times you difference $y_t$ (i.e. work with $y_t - y_{t-1}$, or the second difference) before the series is stationary. You determine d with a stationarity test (ADF/KPSS) rather than by eye: difference until the test says stationary, and stop as soon as it does. Over-differencing (d too large) introduces artificial negative autocorrelation and inflates forecast variance.
• MA(q): today's value depends on the last q forecast errors, not raw values. An ACF that cuts off sharply after lag q (while the PACF decays slowly) points to an MA(q) term.
• Order identification in practice: plot ACF and PACF of the differenced series. AR signature = PACF cuts off, ACF tails off. MA signature = ACF cuts off, PACF tails off. Mixed ARMA signatures (both tail off) are common in practice, so analysts usually also compare a small grid of candidate (p,d,q) by AIC/BIC rather than trusting the plots alone.
• When to add seasonal terms (SARIMA): if, after taking the ordinary difference, the ACF/PACF still show significant spikes at the seasonal period (e.g. lag 7 for daily-with-weekly-seasonality data, lag 12 for monthly-with-yearly-seasonality), a plain ARIMA hasn't captured the seasonal structure. SARIMA adds seasonal AR/MA/differencing terms $(P,D,Q)_s$ operating at multiples of the seasonal period s, on top of the ordinary $(p,d,q)$ terms.

Worked example
Take daily retail sales with a weekly pattern. Fitting an ARIMA(1,1,1) leaves a residual ACF with a clear spike at lag 7 (and 14, 21) - the model has removed the trend (via d=1) but not the weekly repetition. Adding a seasonal term, e.g. SARIMA(1,1,1)(1,0,1)$_7$, lets the seasonal AR/MA terms absorb that lag-7 structure; after refitting, the residual ACF should show no significant spikes at multiples of 7. Concretely: raising d from 0 to 1 changes what "capturing p and q" even means, since AR(p)/MA(q) are now fit on the differenced series, not the raw one - a common early mistake is reading ACF/PACF on the raw series and getting orders that don't apply once differencing is applied.

Trade-offs & pitfalls
Adding seasonal terms multiplies the number of parameters and the search space (p,d,q,P,D,Q,s); over-specifying any one of them (especially D, seasonal differencing) can remove real signal along with the seasonality. A senior candidate will also flag that pure ACF/PACF reading is a starting point, not a final answer: automated order search guided by AIC/BIC (or pmdarima.auto_arima) is standard practice once the visual signature is ambiguous, and the final choice should always be checked with a residual diagnostic (no significant autocorrelation left, e.g. a Ljung-Box test) rather than trusted from the identification step alone.

Direct answer
For intermittent demand, RMSE and MAPE both perform poorly (MAPE is undefined at the many zero periods, and RMSE is dominated by the rare nonzero spikes rather than reflecting typical performance); use metrics designed for intermittent series (MASE against a seasonal-naive-of-zero baseline, or specialized metrics like the Periods-In-Stock accuracy), and modeling approaches purpose-built for lumpy, zero-heavy demand - Croston's method, TSB, or a probabilistic count model - rather than standard ARIMA/ETS.

Structured elaboration

• Why RMSE/MAPE fail here: MAPE divides by the actual value, which is frequently exactly zero in intermittent series, making the metric literally undefined for most periods; RMSE's squared-error penalty means the handful of large nonzero-demand spikes dominate the score entirely, effectively ignoring how well the model does on the much more common all-zero periods, which is often what actually matters operationally (avoiding both stockouts on the rare demand event AND excess holding cost during the many zero periods).
• Croston's method: separately estimates (a) the average SIZE of demand when it does occur and (b) the average INTERVAL between demand occurrences, then combines them into a forecast - rather than trying to smooth a series that's mostly zero, it decomposes the problem into "how much, when it happens" and "how often it happens."
• TSB (Teunter-Syntetos-Babai): a refinement of Croston that updates the demand-occurrence PROBABILITY every period (even zero-demand periods), rather than only updating on periods with actual demand as classic Croston does - this fixes a known bias in Croston's original formulation where the forecast doesn't decay during a long run of zeros, which can leave forecasts stale for slow-moving items.
• Modified Croston / ML alternatives: various bias-correction variants of Croston exist to address its known small-sample bias; ML alternatives (e.g. a classifier for "will demand occur this period" combined with a regressor for "how much, given it occurs") can incorporate additional features (promotions, seasonality) that classic Croston/TSB cannot.
• Probabilistic methods: rather than a single point forecast, model the full demand-count distribution directly - a zero-inflated Poisson or Negative Binomial regression (or a bootstrap/empirical distribution over historical demand-per-period) that returns P(demand = k) for each future period instead of just an expected value. This matters specifically because the decision an intermittent-demand forecast actually feeds (safety stock, reorder point, a target service level) is a tail-risk/stockout-probability question, not a point-accuracy question - a probabilistic model that outputs the full distribution supports that decision directly, where a Croston/TSB point forecast alone needs a separate, bolted-on distributional assumption to get there.
• Evaluation metrics for intermittent series: MASE (against a seasonal-naive baseline, since its denominator - the naive forecast's own error - remains well-defined even with many zeros) is a standard scale-independent choice; specialized metrics that directly weigh stockout risk and holding cost asymmetrically (rather than a symmetric error metric) are often more operationally meaningful, since under- and over-forecasting intermittent demand usually have very different real costs.
• Aggregating metrics across many SKUs for business reporting: report the DISTRIBUTION of per-SKU MASE (median, and the tail of poorly-performing SKUs) rather than a single blended average, since a single number obscures whether errors are broadly small or concentrated in a few problematic SKUs that need individual attention; volume- or value-weighting the aggregate additionally reflects business impact rather than treating every SKU equally regardless of size.

Worked example
A SKU with demand in only 30% of weeks (zero the other 70%) forecast with classic Croston would produce a constant, non-zero expected-demand-per-period estimate blending the "how much" and "how often" components - useful for driving an average reorder policy, but the TSB variant's per-period-updated occurrence probability would react faster if that SKU's demand frequency itself started trending down (e.g. a slow-moving item becoming even slower), which classic Croston's occurrence-only-updates-on-demand-periods formulation would be slower to reflect.

Trade-offs & pitfalls
Applying a standard ARIMA/ETS model to intermittent demand without recognizing the zero-heavy structure typically produces a forecast that's neither a good point estimate for typical (zero) periods nor useful for planning around the rare demand events - the model ends up representing neither case well. Always check the fraction of zero periods in a series before defaulting to a standard forecasting approach; a meaningful zero-fraction (rule of thumb: above roughly 30-40%) is the signal to switch to an intermittent-demand-specific method.