Spotify Data Scientist (Senior Level) Interview Preparation Guide

Direct answer

A cron job only knows a clock time; it has no idea whether the upstream stage it depends on actually finished, failed, or is still half-written, so a multi-stage pipeline built on cron entries is really a set of independent guesses about how long each stage takes. A workflow orchestrator (Airflow, Dagster, Prefect and similar) models the pipeline as a directed acyclic graph (DAG) of tasks with explicit dependency edges, so a downstream task only starts when its upstream task actually reports success, and the system gives you retries, backfills, and one place to see the whole pipeline's health for free.

Structured elaboration

Dependency-awareness versus time-awareness. Cron triggers on wall-clock time alone. If stage two is scheduled 30 minutes after stage one, that gap is a guess about stage one's runtime, not a guarantee. An orchestrator's DAG edge says "task B runs after task A succeeds," which is true regardless of how long A actually takes that day.

Failure propagation. In a DAG, if a task fails, its downstream dependents simply do not run (or run a defined failure branch), automatically. With cron, each stage fires on its own schedule no matter what happened upstream, so a failed stage one silently lets stage two start against stale or partial data.

Retries with policy. Orchestrators let you attach a retry policy per task (bounded attempts, backoff), so a transient failure recovers without a human noticing. A cron job that fails just waits for its next scheduled fire, however far away that is.

Backfill and replay. Because tasks are parameterized by the DAG's logical run (a date or partition), you can rerun a single historical slice through the whole dependency chain. Cron has no concept of "this stage, for that date," so backfilling means manually re-invoking scripts in the right order.

Observability. A DAG-based orchestrator gives one UI or API surface for task duration, success/failure history, and where in the chain something is stuck. Cron gives you whatever each script happens to log, scattered across machines and cron logs.

When cron is still the right tool. A single, independent job with no downstream dependents and no need for backfill or retry policy does not need a DAG orchestrator; adding one there is pure overhead.

Worked example

Take a 3-stage pipeline: extract, transform, load. With cron, each stage gets a fixed offset:

This assumes extract always finishes within 30 minutes. If the source is slow once and extract takes 45 minutes, transform still fires at 01:30 and reads a half-written file; cron has no signal that would stop it. Generalizing to a chain of $N$ stages, cron needs $N$ independently tuned fixed offsets, each one a manual guess that has to be re-tuned whenever runtime varies:

For $N = 5$: cron requires 5 separately maintained time guesses; the DAG requires 4 edges, and those edges stay correct no matter how long any one stage takes, because the trigger is "the previous task instance succeeded," not a clock offset.

Retries make the gap sharper. Suppose a task fails transiently (a brief network blip to the source). An orchestrator retries with exponential backoff, say attempts at 1, 2, and 4 minutes after failure:

A cron-based stage that fails has no retry concept built in: the next opportunity to reprocess is the next scheduled fire, often the next day:

That gap, 7 minutes versus 1440 minutes, is the practical cost of "no retry semantics," not an abstract one.

Trade-offs & pitfalls

An orchestrator is another system to run and keep healthy (unless you use a managed offering), so standing one up for a single independent job with no dependents is pure overhead with no payoff. Sensor or polling patterns (a task that waits by repeatedly checking for a file or a signal) can add their own latency or cost if the poll interval is misconfigured, so "dependency-aware" isn't automatically "instant." A common wrong turn is putting heavy transformation logic directly inside orchestrator tasks instead of having the orchestrator call out to dedicated compute; the orchestrator's job is to sequence and observe work, not to do the work itself.

Direct answer: Streaming quantile estimation (median, 95th percentile, etc.) over data too large to store exactly trades a bounded approximation error for sublinear space: t-digest and Greenwald-Khanna (GK) are two well-known approaches, each maintaining a compact summary whose space scales with the desired error tolerance $\epsilon$ rather than with the number of items seen, and both support merging summaries computed on separate partitions - essential for distributed computation.

Structured elaboration

• Greenwald-Khanna: maintains a sorted list of "tuples" (value, rank-range) summarizing the distribution, with a formal, PROVEN worst-case space bound of $O\left(\frac{1}{\epsilon} \log(\epsilon n)\right)$ for guaranteeing any queried quantile is accurate within $\epsilon$ of its true rank (as a fraction of total items seen) - a genuine worst-case guarantee, not merely an empirical one, which is GK's main appeal for applications needing provable bounds.
• t-digest: clusters nearby values into a bounded number of centroids, allocating MORE, smaller clusters near the distribution's tails (where quantile accuracy usually matters most, e.g. p99, p999) and fewer, larger clusters near the middle - giving especially good accuracy for extreme percentiles at a given memory budget, though without as clean a worst-case theoretical bound as GK; t-digest's practical accuracy is typically excellent and it has become the more commonly deployed choice in modern observability/monitoring systems (e.g. it's the default quantile-sketch in several widely-used metrics systems).
• Mergeability: both support combining two summaries (computed on disjoint data shards) into one representing the union, without re-scanning the original data - t-digest merges by combining and re-clustering centroids; GK's mergeability is more involved but achievable. This property is what makes either approach usable in a distributed aggregation pipeline (compute per-shard, merge centrally).

Trade-offs & pitfalls

• GK's worst-case bound is attractive when you need a provable guarantee (e.g. for an SLA or compliance requirement stating "our reported p99 is accurate to within X"), but its implementation and merge logic are more complex than t-digest's.
• t-digest's tail-weighted accuracy (better precision for extreme percentiles like p99.9 than for the median) matches most real observability use cases, where the tail is exactly what engineers care about most, but its accuracy guarantee is empirical/practical rather than a hard worst-case bound.
• Neither structure gives EXACT quantiles - if a use case genuinely needs an exact median (e.g. a billing calculation), no sketch is appropriate; sketches are for observability/monitoring contexts where a tight approximation is an acceptable and necessary trade for the memory savings.

Process: 1) Align on business objective and constraints — convene marketing and finance to enumerate objectives, timelines, and acceptable trade-offs. 2) Quantify impacts — translate each KPI into business metrics (e.g., marketing cares about conversion lift; finance cares about margin/ROI). Run quick simulations showing how optimizing for one metric affects the other. 3) Propose a multi-objective approach — either a weighted objective with agreed weights, a constrained optimization (maximize marketing KPI subject to finance threshold), or separate models for different audiences with a governance layer.
Decision: Recommend the constrained approach when finance limits are hard (e.g., CAC cap), or a Pareto analysis and pilot A/B tests if uncertainty exists. Always document chosen objective, expected trade-offs, and monitoring plan to detect undesirable drift.

Direct answer: Naming collisions across teams (two features with a similar name and a subtly different definition, or two teams both computing something like "user_engagement" differently) are best prevented with a required namespace convention plus automated similarity detection at review time, and resolved when they do occur through an explicit consolidation-or-alias governance flow rather than letting both versions silently coexist.

Structured elaboration:

• Namespacing: require every feature name to be qualified by its owning team or domain (e.g. growth.engagement_score versus retention.engagement_score), which doesn't prevent semantic collisions but at least makes it structurally impossible for two teams' features to literally collide on the same bare name.
• Metadata strategy: every feature's catalog entry should include enough of its actual computation logic (not just a free-text description) that an automated similarity check can compare two candidates' underlying logic, not just their names.
• Automated similarity detection: at feature-review time, compare a new feature's name, description, and transformation logic against the existing catalog (textual similarity on descriptions, structural similarity on the underlying SQL/code, or distributional similarity on computed values) and flag likely duplicates for human review before they ship.
• Governance flow for resolution: when a genuine collision is found (two features that ARE measuring the same underlying concept), the resolution is either to consolidate onto one canonical feature with the other becoming a deprecated alias, or, if they're subtly-but-genuinely different, to rename both so the distinction is explicit in the name itself, never to just leave two ambiguously-named things both live.

Worked example: Two teams each independently build a feature they call user_engagement: one computes it as a weighted sum of session count and average session duration, the other as a binary "active in the last 7 days" flag. A downstream analyst pulling user_engagement without knowing which team's version they got would draw completely different conclusions depending on which table they queried; an automated review-time similarity check on the NAME alone would have flagged this collision before either feature shipped, giving the teams a chance to either genuinely consolidate or rename to growth.engagement_weighted_score and retention.active_7d_flag so the ambiguity can't happen silently.

Trade-offs and pitfalls: Namespacing alone doesn't solve the deeper problem: two features can have completely different names and still measure the same underlying concept (the semantic collision the automated similarity check is meant to catch), or the same name and genuinely mean different things by design (in which case the fix is a documentation/naming clarity problem, not a technical dedup problem).

Direct answer

Three moves turn a jargon sentence into something an executive can act on: lead with the business outcome instead of the mechanism, swap engineering verbs for plain ones, and reach for an analogy only when it doesn't overstate what's actually guaranteed. None of that means removing information, it means reordering it so the part the executive needs to decide on comes first.

Structured elaboration

1. Lead with outcome, not mechanism. State the risk, cost, or benefit first, then attach the technical action as the "how," not the headline.
2. Replace engineering verbs with plain ones. "Rotate," "provision," "deploy" mean nothing to someone outside engineering; "renew," "set up," "roll out" carry the same meaning without the vocabulary tax.
3. Use an analogy only when it survives a follow-up. An analogy that implies a stronger guarantee than the system actually provides, calling an eventually-consistent system "instant," will bite you the first time it breaks in front of the audience.
   These three moves aren't limited to a single sentence. The same reordering scales to a longer live session, for example a technical workshop script: open with the business outcome for the whole session, and only layer in the underlying mechanism as the audience asks for it, rather than front-loading the architecture before anyone hears why it matters to them.

Worked example

Jargon: "We need to rotate TLS certificates and update our ingress controllers."
Executive version: "We need to renew a security certificate before it expires, and update the component that routes incoming traffic to our services, so customer connections stay encrypted and the site doesn't go down when the old certificate lapses."
Why it's better: the executive version leads with the two things that matter to a non-engineer, security and uptime, keeps the concrete nouns (certificate, routing) but strips the internal name ("ingress controller"), and states the consequence of not acting, the site goes down, instead of leaving the urgency implicit in "we need to."

Trade-offs and pitfalls

Over-simplifying into a metaphor that implies a false guarantee is worse than leaving a term untranslated, because it sets an expectation you can't meet. Calling a best-effort backup "instant recovery" is the kind of thing that gets quoted back to you during an actual incident. Stripping out every technical noun can also read as evasive: "we made some changes" invites more scrutiny than naming the certificate and the routing layer, which sound concrete and controlled. The goal is removing vocabulary that requires domain training, not removing the substance of what changed.

Direct answer. Accept the less-readable version only when you have MEASURED evidence the hot path actually needs it (not a guess), and only alongside documentation, tests, and isolation that keep the cost contained to exactly that one function -- readability should be the default everywhere else in the same codebase.

What would make me accept the trade-off

• Profiling evidence, not intuition: a flamegraph or benchmark showing this specific function is a measurable bottleneck under realistic load, not a guess that 'this looks slow.'
• The gain is worth the readability cost: a 2x improvement on a function called millions of times per second matters; a 2x improvement on something called once a minute doesn't justify the trade at all, regardless of how 'hot' the code technically is.
• No simpler alternative gets you there: readability-preserving options (a better algorithm, caching, batching) have been considered and don't close the gap -- micro-optimization is the last resort, not the first idea.

What I'd require alongside the less-readable version

1. A comment explaining WHY, with the profiling evidence referenced (a link to the benchmark or flamegraph), so a future reader doesn't 'clean up' the optimization back into the slower, clearer form without realizing why it exists.
2. A test that pins the PERFORMANCE characteristic, not just correctness -- so a future change that silently reintroduces the slow path is caught, not just a change that breaks output.
3. Isolation: the optimized code lives behind a clearly-named function/module boundary with a normal, readable interface, so the complexity is contained to one place rather than leaking into every caller.
4. A note in the PR/commit explicitly flagging the trade-off for reviewers, so it's a deliberate, reviewed decision rather than something that slips through because 'the benchmark numbers looked good.'

A worked framing

Say a hashing function is called on every request in a load-balancer's hot path. Profiling shows a straightforward, readable implementation costs 40ns per call versus 8ns for a bit-twiddling optimized version, and this function alone accounts for 15% of total CPU under peak load. That's worth it: document the bit-twiddling with WHY, pin the benchmark in CI, and keep the optimized version behind a small, well-named function (fast_hash()) so callers still read cleanly.

Trade-offs and pitfalls

• The single most common mistake is skipping the measurement step and optimizing based on intuition -- many 'obviously slow' patterns turn out to be irrelevant to actual bottlenecks once profiled, and the readability cost was paid for nothing.
• Don't let one legitimately-optimized hot path set a PRECEDENT that spreads unmeasured 'clever' code elsewhere in the codebase under the banner of 'performance matters here too' -- keep the exception genuinely exceptional and visibly justified.

Direct answer

Build the cost model around one variable: the fraction of time the workload spends at the spike rate, because that single number determines whether on-demand (pay per request) or provisioned (pay for reserved throughput) billing is cheaper, and solve for the break-even point rather than guessing. For the stated shape (baseline 100 requests/sec, spikes to 10,000 requests/sec for 5% of the time), on-demand billing wins clearly against provisioning for the peak, and the break-even doesn't tip toward provisioning until spikes occupy roughly 28% of the time, using DynamoDB's current, verified list rates as the concrete pricing model. A self-managed alternative (Cassandra, an open-source distributed database you install and operate yourself instead of paying a managed service like DynamoDB per request) removes the per-request pricing question entirely but replaces it with a fixed infrastructure-plus-staffing cost that needs the same kind of break-even comparison against both DynamoDB options.

Variables to include

• Baseline and spike request rates (R_base, R_spike) and the fraction of time at spike (f). These three fully determine total request volume.
• On-demand unit price: dollars per million requests, separately for reads and writes if the workload mix isn't symmetric.
• Provisioned unit price: dollars per provisioned-capacity-unit-hour, and a strategy for how much capacity to provision (for peak at all times, versus autoscaled).
• Autoscaling lag and cost: provisioned autoscaling reacts on a delay (commonly several minutes for a managed service), so "provision for baseline and autoscale to peak" is only realistic for spikes that are predictable or gradual, not sudden bursts.
• Storage and fixed costs: additive to both models, not part of the break-even math itself.

Cost formulas

where $p_{wr}$ is the on-demand price per million write-request-units and $p_{wcu}$ is the provisioned price per write-capacity-unit-hour. Setting $C_{ondemand}(f) = C_{peak}$ and solving for $f$ gives the break-even spike fraction:

Worked example, using DynamoDB's verified current rates

Current AWS list prices (aws.amazon.com/dynamodb/pricing, US East N. Virginia, checked 2026-09-24): on-demand writes at $0.625 per million write-request-units, and provisioned writes at $0.00065 per write-capacity-unit-hour. Assume each write is a small item consuming one unit.

base_rps, spike_rps, f_spike = 100, 10_000, 0.05
seconds_per_hour = 3600
on_demand_rate = 0.625        # $/million WRU, VERIFIED current AWS list price
provisioned_rate = 0.00065    # $/WCU-hour, VERIFIED current AWS list price

writes_per_hour = seconds_per_hour * ((1 - f_spike) * base_rps + f_spike * spike_rps)
on_demand_cost = writes_per_hour / 1_000_000 * on_demand_rate
provision_for_peak = spike_rps * provisioned_rate
autoscale_ideal = ((1 - f_spike) * base_rps + f_spike * spike_rps) * provisioned_rate

print(f"writes/hour: {writes_per_hour:,.0f}")
print(f"on-demand: ${on_demand_cost:.4f}/hr")
print(f"provisioned for peak: ${provision_for_peak:.4f}/hr")
print(f"provisioned, perfect autoscale (optimistic, ignores scale-up lag): ${autoscale_ideal:.4f}/hr")

a = seconds_per_hour / 1_000_000 * on_demand_rate
f_star = (provision_for_peak - a * base_rps) / (a * (spike_rps - base_rps))
print(f"break-even spike fraction (on-demand vs. provision-for-peak): {f_star:.3f}")

writes/hour: 2,142,000
on-demand: $1.3387/hr
provisioned for peak: $6.5000/hr
provisioned, perfect autoscale (optimistic, ignores scale-up lag): $0.3867/hr
break-even spike fraction (on-demand vs. provision-for-peak): 0.282

At the stated 5% spike fraction, on-demand costs $1.34/hr versus $6.50/hr for provisioning capacity that always covers the peak: on-demand is roughly 4.9x cheaper. Provisioning only becomes cheaper than on-demand once spikes occupy about 28% of the time, nearly six times the stated workload's actual spike frequency. The idealized-autoscale number ($0.39/hr) looks best of all, but it assumes the scale-up completes before the spike arrives, which for DynamoDB's managed autoscaling (which reacts on a multi-minute delay) is only realistic for spikes the system can see coming, not a sudden, unannounced burst; a genuinely bursty, unpredictable spike either needs on-demand billing or provisioned capacity sized well above baseline as a safety margin, not the idealized number above.

Where self-managed Cassandra fits

A self-managed Cassandra cluster sidesteps DynamoDB's per-request or per-capacity-unit pricing entirely and instead has a mostly fixed infrastructure cost (nodes sized for the spike, since Cassandra has no serverless mode) plus the ongoing staffing cost of running and tuning a distributed database. That comparison isn't a pricing-formula problem the way the two DynamoDB modes are; it's the same kind of staffing-dominated TCO (total cost of ownership) trade-off as build-versus-buy for an analytics warehouse: cheaper per-request at high, sustained volume, but only once the fixed infrastructure-plus-staffing cost is spread over enough traffic to beat DynamoDB's on-demand rate, and only if the team already has (or is willing to build) the operational expertise to run Cassandra's repair, compaction, and multi-datacenter replication correctly.

Operational trade-offs

• On-demand (pay-per-request): cost tracks actual usage exactly, zero capacity planning, but per-unit price is highest and a traffic anomaly (including a bug that causes a request storm) is directly a cost incident, not just a performance one.
• Provisioned, sized for peak: predictable bill, no throttling risk, but pays for idle capacity 95% of the time in this workload's shape.
• Provisioned with autoscaling: closer to on-demand's cost efficiency, but only for spikes the system has time to react to; a genuinely sudden burst risks throttled requests during the scale-up lag, which is a correctness/availability risk, not just a cost one.
• Self-managed (Cassandra): removes per-request pricing risk entirely, but trades it for a fixed staffing and infrastructure commitment that only pays off at sustained high volume.

Trade-offs and pitfalls

• Don't provision for peak "just to be safe" without running this arithmetic. At a 5% spike fraction, that default choice costs roughly 5x more than on-demand for no correctness benefit.
• Autoscaling's "ideal" cost number is optimistic by construction. Report it alongside its assumption (scale-up completes before the spike hits), not as the expected real-world cost.
• A single-parameter model (spike fraction) is a starting point, not the whole story. Real traffic often has multiple spike tiers and diurnal patterns; the same break-even technique extends to a piecewise model, but the two-tier version above is enough to make the on-demand-versus-provisioned decision for this stated workload.

Direct answer
A strong answer separates what failed from why the candidate personally missed it, shows visible triage speed, states plainly how stakeholders were told, and ends with one lesson that changed a later, verifiable outcome rather than a general resolution to try harder.

Situation, discovery, triage, communication, and lesson

• Situation: I built a lead-scoring model meant to help sales prioritize which leads to call first. Offline evaluation showed good separation between high- and low-scored leads.
• Discovery: I started pulling a weekly conversion-by-decile report after launch and noticed the top decile's conversion rate wasn't meaningfully different from the bottom decile's, the model had no real discrimination once it was live.
• Immediate triage, practical and emotional: my first instinct was to quietly dig for the bug before telling anyone. I caught myself doing that and reversed it: within the same day I froze the model as the ranking signal, reverting sales to the prior manual heuristic, so the team stopped wasting calls on a broken ranking while I investigated. Admitting that quickly, before I had an explanation, was more uncomfortable than waiting until I had a tidy story, but the alternative was letting sales keep trusting a signal I already knew was hollow.
• Communication: I sent a short note the same day to the sales lead and my manager: what's paused, what isn't affected, and when to expect an update, in plain numbers rather than model language.
• Root cause and the reasoning gap: the offline evaluation had used historical outcomes that were themselves a product of the sales team's existing judgment-based prioritization, so the model had partly learned to reproduce the old heuristic rather than improve on it, a circularity my offline metric couldn't see. My own miss was skipping a randomized online pilot before full rollout; I trusted the offline number too far.
• Lesson implemented: I made a small-traffic randomized pilot mandatory before any new scoring model fully replaces an existing prioritization process, specifically to catch this exact confound. The next scoring project I ran went through that pilot gate, and the pilot did catch a similar circularity before full rollout, which is the concrete evidence the change held.

Weak vs. strong answer
A weak answer says "the model underperformed, so we retrained it with more data," without naming the specific evaluation flaw. A strong answer names the exact circularity in the ground truth and points to a governance change, the mandatory pilot, that a later project actually went through.

Trade-offs and pitfalls
Requiring a pilot for every model change is costly if a team ships frequently; it's worth being explicit that this applies to changes that alter a real decision (like which lead gets called), not low-stakes internal-only models. A pilot also only catches this specific circularity failure mode, not every bug class, so it shouldn't be presented as a complete safety net on its own.

Direct answer

The underlying finding stays identical, but you change altitude, vocabulary, and level of supporting detail. An executive gets the headline, the business impact, and the recommended decision in one or two lines up front. The implementation team gets the mechanism, the caveats, and enough of the underlying data to act on it correctly.

Structured elaboration

• Altitude: conclusion-first for the executive, versus enough method detail for the team to trust and reproduce the diagnosis.
• Vocabulary: business-impact language (revenue, risk, timeline) for the executive, technical specifics (segments, funnels, thresholds) for the team.
• Format: a one-slide or one-paragraph summary versus a working document with a data appendix.
• What must never change: the number itself and the direction of the conclusion, in both versions.

Worked example

Finding: onboarding drop-off at step 3 is costing an estimated 6% of new signups per month.
Executive version: "we're losing about 6 of every 100 new signups at the step-3 confirmation screen, fixing it could recover meaningful revenue this quarter, recommend prioritizing it."
Team version: "62% of that drop-off happens on mobile between form submit and confirmation render, median time to abandon is 9 seconds, this looks like a loading-state issue on mobile specifically."
Both versions agree on the 6% headline number and the recommendation to prioritize the fix.

Trade-offs and pitfalls

The two versions can quietly drift into different conclusions if you're not careful, always trace both back to the same underlying analysis. Over-simplifying for the executive can also strip out the one caveat that would have changed their decision, so pick what to omit deliberately, not by default.

What the interviewer probes next

Expect a question about what happens when the executive summary gets forwarded on without you in the room, and how you prevent it from being read out of context.

Validation set: a dataset held out from training used to compare models and tune hyperparameters. It guides model selection but influences choices, so it can cause optimistic bias if reused too much. Test set: a final, untouched dataset used only once (or very rarely) to estimate true generalization performance after all decisions are locked.

Common pitfalls and leakage sources:

• Peeking at the test set (using test metrics to choose models/hyperparams).
• Target leakage: including features derived from future information or the label (e.g., using "days_until_readmission" to predict readmission).
• Preprocessing leakage: fitting scalers, imputers, PCA, or feature selection on the full dataset instead of only the training split.
• Time-series misuse: random splits that mix past/future, leaking future information into training.
• Reusing validation repeatedly without correction: overfitting to the validation set.

Practical rules and pipeline structure to avoid issues:

• Rule 1: Never touch the test set until final evaluation.
• Rule 2: Always fit preprocessing (imputer, scaler, encoder, feature selection, PCA) only on training data; apply the fitted transformers to validation/test.
• Rule 3: Use scikit-learn Pipelines (or equivalent) so transforms and model fitting are atomic and safe inside cross-validation.
• Rule 4: For hyperparameter tuning, use cross-validation on the training set or nested CV: outer loop for performance estimate, inner loop for hyperparameter selection.
• Rule 5: For time-series, use time-aware splits (e.g., rolling/expanding window) to preserve temporal order.
• Rule 6: Put feature engineering that uses the label or aggregates across rows (e.g., group-based stats) inside the training fold computation to avoid leakage.
• Rule 7 (group splits): When records are grouped by an entity, for example every event from the same user or session, the whole group must land in only one split. A random row-level split can put 8 of a user's 10 events in train and the other 2 in test, so the model has effectively already seen that user during training and the test score is optimistically inflated. Use GroupKFold (or an equivalent grouped split) keyed on user_id or session_id so no single user's or session's rows ever appear in both train and test.
• Rule 8 (stratification): A plain random split can, by chance, land a rare class at very different rates across train/val/test, for example a 2% fraud rate in training but 0.5% in the test fold purely from sampling noise. Stratified splitting (e.g., scikit-learn's StratifiedKFold or a stratified train_test_split) preserves the proportion of each class across every split, so a rare class doesn't vanish from or get amplified in one fold by chance, and validation metrics reflect the model's actual behavior on the true class ratio.

Example pipeline (sketch):

• Split: train / validation / test (or train+CV / test)
• Within CV fold or training split:
  • Fit imputer → fit scaler/encoder → fit feature selector/PCA → train model
  • Apply same fitted transforms to validation/test
• For hyperparameter search: use GridSearchCV with Pipeline or nested CV

These practices ensure preprocessing and selection decisions are based only on training information, preventing leakage and giving reliable estimates of real-world performance.