Forecasting and Time-Series Analysis Questions

Direct answer
A sensitivity analysis for a 5-year strategic forecast means systematically varying key drivers (price, volume growth, churn) across plausible ranges, computing how much the forecast output moves per unit change in each driver (an elasticity), visualizing the biggest drivers with a tornado chart, and running a joint Monte Carlo simulation to show the combined uncertainty rather than treating each driver in isolation.

Structured elaboration and worked example (executed)
Three correlated drivers - leads, conversion rate, deal size - set up so their means multiply to a base monthly revenue of $100,000, with leads/conv_rate negatively correlated (larger, lower-quality lead volume tends to convert worse) and leads/deal_size mildly positively correlated:

import numpy as np

rng = np.random.default_rng(42)
n_sims = 100_000

means = np.array([1000.0, 0.10, 1000.0])        # leads, conv_rate, deal_size
stds  = np.array([150.0, 0.016, 125.0])         # ~15%, ~16%, ~12.5% CV

corr = np.array([
    [1.0, -0.3, 0.1],
    [-0.3, 1.0, 0.0],
    [0.1,  0.0, 1.0],
])
L = np.linalg.cholesky(corr)
z = rng.standard_normal((n_sims, 3))
draws = means + (z @ L.T) * stds             # correlated draws across 3 drivers

leads, conv_rate, deal_size = draws[:, 0], draws[:, 1], draws[:, 2]
revenue = leads * conv_rate * deal_size

p5, p50, p95 = np.percentile(revenue, [5, 50, 95])
emp_corr = np.corrcoef(draws.T)

base = means[0] * means[1] * means[2]
def one_at_a_time(idx):
    lo, hi = means.copy(), means.copy()
    lo[idx] -= stds[idx]; hi[idx] += stds[idx]
    return lo[0]*lo[1]*lo[2] - base, hi[0]*hi[1]*hi[2] - base

Executed result (numpy, seed=42, 100,000 draws):

P5=$64,535  P50=$97,662  P95=$140,608   (90% interval width: $76,072)
Empirical corr(leads, conv_rate):   -0.3017  (target -0.3)
Empirical corr(leads, deal_size):    0.1049  (target 0.1)

and a tornado-style one-at-a-time sensitivity, moving each driver by ±1 standard deviation from its mean with the others held fixed:

leads:      [-15,000, +15,000]   (base=$100,000)
conv_rate:  [-16,000, +16,000]
deal_size:  [-12,500, +12,500]

Here conversion rate has the largest one-at-a-time sensitivity of the three, making it the natural top-priority driver for deeper investigation.

• Choosing driver ranges: base each range on genuinely observed historical variability (e.g. ±1 standard deviation of the driver's own historical movement), not an arbitrary or hopeful range - an unrealistically narrow range understates true risk, an unrealistically wide one produces an unhelpfully vague forecast.
• Computing elasticities: the one-at-a-time deltas shown above ARE a simple elasticity measure (revenue change per driver's own typical variation); a more formal elasticity (percent output change per percent input change) is the same idea normalized to be comparable across drivers with very different natural scales.
• Tornado charts: rank the one-at-a-time sensitivities by magnitude and display as horizontal bars from largest to smallest impact, giving stakeholders an immediate visual read on which drivers matter most, without needing to interpret raw regression coefficients or simulation output directly.
• Monte Carlo for joint uncertainty: the one-at-a-time tornado view holds every OTHER driver fixed, which understates the true combined range once drivers move together and interact (as demonstrated by their correlation, verified above) - the Monte Carlo simulation captures this joint uncertainty properly, which is why both views are useful together: tornado charts for prioritization, Monte Carlo for the honest combined uncertainty range.
• Prioritizing drivers for deeper analysis: rank by tornado-chart impact magnitude first, but ALSO weigh how genuinely uncertain (versus already well-understood) each driver currently is - a high-impact driver you already have strong historical grounding for needs less further work than a similarly high-impact one you're still genuinely unsure about.

Trade-offs & pitfalls
A tornado chart alone, by design, ignores correlation and interaction between drivers - presenting only the one-at-a-time view without the joint Monte Carlo range risks understating (or in some correlation structures, overstating) the true combined uncertainty; always pair the two rather than relying on either alone. And any sensitivity analysis is only as credible as the assumed driver ranges and correlation structure - document those assumptions explicitly, since a stakeholder's confidence in the whole exercise should track how well-grounded those inputs actually are, not just the sophistication of the simulation itself.

Direct answer
Additive seasonality assumes the seasonal swing is a roughly constant absolute amount regardless of the series' level; multiplicative seasonality assumes it scales proportionally with the level. Choosing the wrong one distorts both baselines (an additive model under-predicts the seasonal swing at a high level, or over-predicts at a low one) and anomaly thresholds (a fixed-width band around an additive baseline will be too tight during high-level periods and too loose during low-level ones).

Structured elaboration

• Additive: $y_t = \text{trend}_t + \text{seasonal}_t + \text{residual}_t$ - the seasonal swing stays roughly the same size in absolute units no matter what the trend level is doing.
• Multiplicative: $y_t = \text{trend}_t \times \text{seasonal}_t \times \text{residual}_t$ - the seasonal swing grows or shrinks proportionally with the trend level; a multiplicative model can be fit as an additive one on the LOG of the series, since $\log(y_t) = \log(\text{trend}_t) + \log(\text{seasonal}_t) + \log(\text{residual}_t)$.
• Which metrics favor which: revenue-style metrics that grow substantially over time, where the seasonal spike naturally grows alongside the business (e.g. a December bump that's proportionally consistent but has grown from $10K to $50K as the business scaled), are usually multiplicative. Metrics with a roughly stable base level and a genuinely fixed-size seasonal effect (e.g. a fixed number of extra support tickets every Monday, regardless of overall ticket volume trend) are more often additive.
• Testing which model fits better: the simplest visual test is whether the peak-to-trough seasonal amplitude visibly GROWS alongside the trend on a raw plot - if it does, additive is the wrong assumption. More formally, fit both, compare residual variance (an additive fit on a genuinely multiplicative series will show residual variance that itself grows with the level, a clear diagnostic on a residual-vs-fitted plot), or simply compare backtested forecast error between the two.
• Why the choice matters for baselines/anomaly detection: an anomaly-detection threshold built on additive-model residuals (e.g. "flag anything more than 3 residual-standard-deviations away") implicitly assumes constant residual variance across the whole series; if the true process is multiplicative, that fixed threshold will be too sensitive (false positives) during low-level periods and too insensitive (missed real anomalies) during high-level periods.

Worked example
Two metrics side by side: daily active users growing from 10,000 to 100,000 over two years, with a weekend dip that's always been roughly 15% below the weekday average - that's multiplicative (the absolute size of the dip has grown 10x alongside the user base, but the RATIO has stayed constant). A metric like "number of scheduled maintenance windows per week," with a fixed operational cadence unrelated to overall traffic growth, is more likely additive.

Trade-offs & pitfalls
Log-transforming to convert a multiplicative problem into an additive one is convenient (lets you reuse additive-model tooling) but changes how you have to interpret and back-transform prediction intervals - a symmetric interval in log-space becomes an ASYMMETRIC interval once exponentiated back to the original scale, which is correct behavior (uncertainty genuinely should be asymmetric on a multiplicative series) but is easy to get wrong if you forget to back-transform properly.

Direct answer
To decide whether a sudden drop is expected seasonal/random variance or a real causal effect from a recent change, build an explicit statistical comparison: model what the series was EXPECTED to do this period (via seasonal decomposition, SARIMA, or a Bayesian structural time series model), quantify how far the observed drop is from that expectation in probabilistic terms, and corroborate with a control series unaffected by the change wherever one is available.

Structured elaboration

• Data selection: use enough history to estimate the seasonal pattern reliably (at least a few full seasonal cycles), and be careful to exclude any period that was itself anomalous (a past outage, a past unrelated shift) from the baseline used to estimate "expected" behavior.
• Candidate models for expected behavior: seasonal decomposition (STL or classical) gives a quick expected value plus an implicit residual-based sense of normal variance; SARIMA gives a formal predictive distribution with a prediction interval; Bayesian Structural Time Series (BSTS) is specifically well-suited here because it's designed for causal-impact style analysis - it produces a full counterfactual prediction ("what would the series likely have done without the change") with a credible interval, which is a more direct answer to "is this drop unusual" than a plain decomposition.
• Using a control series: if a comparable, unaffected series exists (an unaffected region, a cohort not exposed to the deployment), compare its behavior over the same window - if the control ALSO shows a comparable drop, that's strong evidence the true cause is something shared (broader seasonality, a macro event) rather than the specific change being investigated.
• Hypothesis testing and quantifying confidence: frame it explicitly as a hypothesis test - $H_0$: the observed value is consistent with the model's predictive distribution (i.e. explainable by normal variance); if the observed drop falls well outside the model's prediction interval (or, in a BSTS framing, the counterfactual credible interval), reject $H_0$ in favor of a real causal effect, and report the width of that interval so stakeholders understand HOW confident the conclusion is, not just the binary verdict.
• Multiple-cohort correction: if you're checking several cohorts or segments simultaneously for the same kind of drop, correct for multiple comparisons (e.g. a Bonferroni or FDR adjustment on the p-values) - checking 20 cohorts at the standard 5% significance threshold will produce roughly one false "significant" finding by chance alone if left uncorrected.

Worked example
A first practical filter before any formal modeling: distinguish a SUSTAINED trend from a TRANSIENT anomaly with a few concrete checks - does the metric recover within a day or two (favors transient/anomaly) or does the new level persist across multiple full seasonal cycles (favors a real, sustained shift)? Is the drop isolated to one segment (a specific platform, a specific region) consistent with a localized deployment, or does it appear everywhere (favors a shared, external cause like a broad seasonal effect)? Only once these quick checks are ambiguous does the fuller BSTS/control-series analysis earn its cost.

Trade-offs & pitfalls
The most common mistake is skipping straight to "is this significant" without first checking whether the model used to define "expected" was itself well-calibrated (a seasonal model fit on a short or contaminated history will produce an overconfident interval, making ordinary variance look like a dramatic anomaly). Equally common: attributing a drop to the most recent visible change (a deployment) purely because of timing, without checking a control series - correlation in timing alone is weak evidence, and a genuinely rigorous answer needs either a true experiment (if one exists) or a credible quasi-experimental comparison.

Direct answer
Responsible AI and governance for forecasting systems means checking whether the model treats different regions or product lines fairly (not just accurately on average), protecting the privacy of training data, and running the operational governance machinery - model cards, periodic audits, defined remediation processes - that make the whole system auditable and correctable rather than a black box that only gets scrutinized after something visibly goes wrong.

Structured elaboration

• Detecting bias across regions or product lines: check forecast accuracy and, separately, forecast BIAS (systematic over- or under-prediction) broken out BY segment (region, product line), not just in aggregate - a model that's unbiased on average can still systematically under-forecast one region while over-forecasting another, which is invisible to an aggregate accuracy check but very real in its downstream consequences.
• Mitigating detected bias: once a segment-specific bias is confirmed (not just suspected from a single period, but validated the way any bias-detection exercise should be, with a proper statistical check), remediation options range from a segment-specific recalibration correction to retraining with segment-balanced data or segment-aware features, chosen based on WHY the bias exists (a data-representation issue vs a genuine, harder-to-model difference in that segment's underlying dynamics).
• Fairness when forecasts drive allocation decisions: when a forecast feeds directly into an ALLOCATION decision (inventory, staffing, incentive dollars distributed across regions), a systematic forecasting bias against one region translates directly into that region being under-served - this is the concrete mechanism by which a "purely technical" forecasting bias becomes a real fairness issue, and it's the reason bias detection here needs to be checked specifically against WHATEVER downstream decision the forecast drives, not evaluated as an abstract accuracy statistic alone.
• Data retention and privacy: forecasting models trained on customer-level or location-level transaction data inherit the same retention-limits and privacy-handling obligations as any other system using that data - define and enforce a retention policy for training data, and ensure the model itself (and any cached intermediate features) doesn't become an unintended long-term store of data that should have been deleted under the organization's stated retention policy.
• Operational governance practices: model cards (a standardized, versioned document describing a model's intended use, known limitations, training-data characteristics, and validated performance across segments) make a model's fairness/bias characteristics legible to anyone who needs to evaluate or approve its use, rather than requiring a fresh investigation each time; periodic audits (a scheduled, recurring re-check of segment-level bias and accuracy, not just a one-time pre-launch check) catch drift into unfairness that emerges only after deployment; a defined remediation process (what happens, and who's accountable, once an audit finds a problem) ensures a detected issue actually gets fixed rather than merely documented.

Worked example
An automated demand-allocation system that systematically under-forecasts demand in lower-income neighborhoods (perhaps because those areas have historically been served by fewer marketing dollars, and the model has learned that pattern as if it were an accurate reflection of true underlying demand rather than a historical resourcing artifact) would, if left unchecked, perpetuate and even reinforce that under-service through the forecast-driven allocation decision itself - detecting this requires deliberately checking bias BY neighborhood income level (not something an aggregate accuracy metric would surface on its own), and the mitigation needs to address the root cause (the historical resourcing pattern baked into the training data) rather than just a numeric recalibration that treats the symptom.

Trade-offs & pitfalls
The most consequential governance gap is treating fairness/bias review as a one-time, pre-launch checklist item rather than an ongoing, periodically-repeated audit - a model that was checked and cleared at launch can still drift into segment-specific bias over time as the underlying data and business context evolve, and only a recurring audit cadence (not a single point-in-time check) catches that drift before it compounds into a real, sustained allocation harm.

Direct answer
A strong answer to "tell me about a forecast that was significantly off" walks through a genuine situation with a specific root-cause diagnosis, concrete corrective steps (not vague "I learned from it"), and a lasting process or model change - the STAR structure (Situation, Task, Action, Result) is the scaffold, but the substance an interviewer is actually listening for is the diagnostic reasoning and the durability of the fix.

Structured elaboration

• Situation: set up the specific forecast, its stakes, and how far off it turned out to be, with enough concrete detail (what metric, what horizon, roughly how large the miss was) that the story is grounded, not abstract.
• Task: what you were responsible for and what decision the forecast was feeding - this frames why the miss actually mattered, not just that a number was wrong.
• Action - diagnosis: describe the actual investigative process: what hypotheses you considered (a genuine regime change? a data issue? a missed exogenous driver? a modeling assumption that broke?), and how you distinguished between them with evidence rather than guessing - this diagnostic rigor is usually the part that most differentiates a strong answer.
• Action - corrective steps: what you actually changed, concretely - a specific feature you added, a validation gap you closed, a monitoring alert you built - rather than a generic "I retrained the model."
• Action - communicating outcomes to stakeholders: how you told the people who relied on the forecast what happened, proactively rather than waiting to be asked - a concise, honest explanation of the root cause, what you fixed, and what would be different going forward. Stakeholders who feel informed about a miss, rather than blindsided by it, are far more likely to keep trusting the forecast afterward; a technically excellent diagnosis that's never actually communicated to the people who relied on the number doesn't rebuild trust by itself.
• Result: what happened afterward, ideally with some evidence the fix actually worked (a subsequent forecast that held up, a monitoring signal that would have caught the original issue earlier next time) - and, honestly, what you'd still do differently, since acknowledging remaining limitations reads as more credible than claiming a fully solved problem.

Worked example
A credible shape: "our forecast for a key metric ran roughly 15% low for two consecutive weeks; I first ruled out a data pipeline issue (checked the raw counts directly against a source system), then compared against a control series in an unaffected region and found it was ALSO drifting similarly, which pointed away from anything specific to our product and toward a broader, unmodeled seasonal or macro shift; digging further, a competitor had just changed pricing in a way that shifted category-wide demand patterns our model had no way to see. The fix wasn't a bigger model - it was adding a lightweight external signal (category-level demand) as a regressor, plus a monitoring rule that flags when our forecast error and a comparable control series both drift in the same direction, since that combination is a stronger signal of a shared external cause than either alone. I also sent the stakeholders who relied on that forecast a short, proactive write-up: what went wrong, why, what we changed, and what to expect going forward, rather than letting them notice the forecast had quietly improved and wonder what happened." This kind of answer demonstrates genuine diagnostic process (control-series comparison, ruling out alternatives) rather than a post-hoc guess.

Trade-offs & pitfalls
The most common weak answer in this format skips straight from "it was wrong" to "I fixed the model" without showing the actual diagnostic reasoning in between - an interviewer specifically wants to see how you distinguish between competing explanations (a bug, a genuine regime change, a coincidental fluke) with evidence, since that diagnostic discipline is exactly what predicts whether you'll handle the NEXT unexpected miss well, which is the real thing being assessed.