Netflix Data Engineer (Mid-Level) Interview Preparation Guide 2026

Direct answer

Deduplicate on the natural event key (not on the full row) using a deterministic ranking window function, row_number() over partitionBy(event_key).orderBy(ingest_time, tie_breaker), keeping rank 1, rather than dropDuplicates(), because dropDuplicates gives no control over WHICH copy of a duplicate is kept when duplicates disagree in any other column, and late-arriving duplicates specifically need a clear, explicit "first write wins" (or "last write wins", depending on the business rule) policy rather than whatever Spark's internal execution happens to keep.

Structured elaboration

Why not just dropDuplicates(["event_id"])? It works for correctness (exactly one row per event_id survives) but which of several duplicate rows survives is unspecified when they differ in other columns (for example, differing ingest_time due to late arrival, or a corrected payload in a later duplicate); relying on unspecified behavior for a business-meaningful choice (keep the FIRST-seen copy, or keep the copy with the LATEST correction) is fragile and the correct policy varies by domain, so it should be explicit in the query, not implicit in the engine's execution order.

Design for late-arriving duplicates specifically. "Late-arriving" means the SAME logical event can appear again in a LATER micro-batch or a later partition of a batch job, potentially long after the original. A pure in-memory Set-based dedup (as would work for a single bounded batch entirely in memory) does not scale to multi-terabyte data and does not naturally handle duplicates arriving in separate batches days apart; a window-function approach that reads the FULL relevant history (or, for a streaming job, uses dropDuplicatesWithinWatermark bounded by a watermark) is what actually scales.

Shuffle volume. The dedup key's partitioning determines shuffle cost directly: partitionBy(event_id) shuffles the full dataset once (unavoidable, since duplicates of the same key must land in the same partition to be compared), proportional to total data volume, not to the number of DISTINCT keys. For a truly multi-terabyte dataset, this shuffle is the dominant cost of the whole job; there is no way to avoid it while guaranteeing correctness for arbitrarily-late duplicates, since any two duplicate rows could in principle be in different physical files that both need to be compared.

Memory. The window function's per-partition working set is bounded by the number of duplicate ROWS per key (usually small, a handful of copies at most), not by total partition size, so memory pressure here is driven mainly by partition-count/skew choices (the standard shuffle-sizing considerations), not by anything dedup-specific, UNLESS a small number of keys have pathologically many duplicate copies (a genuine data-quality bug upstream, worth alerting on separately).

Correctness under failures. If the dedup job itself fails partway and is retried, re-running the identical deterministic window-function query over the same (or a superset, if incrementally reprocessing) input produces the identical result, since row_number() with a fully deterministic order (including the tie-breaker) is a pure function of its input; this is a meaningfully stronger property than, for example, a stateful streaming dedup relying on an accumulating in-memory or state-store set that must itself be checkpointed correctly to survive a restart without either losing dedup state (letting a duplicate back in) or growing unboundedly.

Worked example

from pyspark.sql import SparkSession
from pyspark.sql import functions as F
from pyspark.sql.window import Window

spark = SparkSession.builder.master("local[2]").appName("dedup").getOrCreate()

data = [
    ("e1", "u1", "2026-01-01 09:00:00", "2026-01-01 09:00:05"),
    ("e1", "u1", "2026-01-01 09:00:00", "2026-01-01 09:00:07"),  # duplicate of e1
    ("e2", "u1", "2026-01-01 09:01:00", "2026-01-01 09:01:02"),
    ("e3", "u2", "2026-01-01 08:50:00", "2026-01-01 09:05:00"),  # late-arriving original
    ("e2", "u1", "2026-01-01 09:01:00", "2026-01-01 09:06:00"),  # very late duplicate of e2
]
df = spark.createDataFrame(data, ["event_id", "user_id", "event_time", "ingest_time"]) \
    .withColumn("event_time", F.to_timestamp("event_time")) \
    .withColumn("ingest_time", F.to_timestamp("ingest_time"))

# First-ingest-wins policy, deterministic tie-break on user_id in case two
# copies share an identical ingest_time.
w = Window.partitionBy("event_id").orderBy("ingest_time", "user_id")
deduped = (df.withColumn("rn", F.row_number().over(w))
             .filter(F.col("rn") == 1)
             .drop("rn")
             .orderBy("event_id"))

deduped.select("event_id", "user_id", "event_time", "ingest_time").show(truncate=False)
print("distinct event_ids:", deduped.select("event_id").distinct().count())
print("row count after dedup:", deduped.count())

Output (actually executed with python3.12 + pyspark 3.5.1, Java 17, local[2]):

+--------+-------+-------------------+-------------------+
|event_id|user_id|event_time         |ingest_time        |
+--------+-------+-------------------+-------------------+
|e1      |u1     |2026-01-01 09:00:00|2026-01-01 09:00:05|
|e2      |u1     |2026-01-01 09:01:00|2026-01-01 09:01:02|
|e3      |u2     |2026-01-01 08:50:00|2026-01-01 09:05:00|
+--------+-------+-------------------+-------------------+
distinct event_ids: 3
row count after dedup: 3

The duplicate e1 copy (ingested at 09:00:07) and the very-late duplicate e2 copy (ingested at 09:06:00, four minutes after the original) are both correctly dropped, keeping the FIRST-ingested copy of each; the late-arriving genuinely-new event e3 (whose event_time is earlier than everything else, but which only arrived at ingest_time 09:05:00) is correctly kept as its own row since it is not a duplicate of anything, demonstrating that "late-arriving" and "duplicate" are independent properties this design handles separately and correctly.

Trade-offs and pitfalls

• Shuffle volume is proportional to total data scanned, not to how sparse the duplicates are. Even a dataset with a 0.01% duplicate rate pays the full shuffle cost of partitioning by event_id, because the dedup logic cannot know in advance which rows are duplicates without comparing them; this is the fundamental cost floor of exact, correctness-guaranteed dedup at scale, and it is not avoidable by a cleverer algorithm, only bounded by scoping the comparison window (see below).
• Bounding the comparison scope for genuinely unbounded lateness. For streaming or very-large incremental batch jobs, comparing every new row against the ENTIRE historical dataset is not sustainable; dropDuplicatesWithinWatermark (Structured Streaming) or an explicit business-defined lateness bound (e.g., "duplicates can arrive up to 7 days late, beyond that treat as a new event") trades perfect correctness for arbitrarily-late duplicates against bounded state size, which is usually the right trade-off in practice since unbounded state growth is itself a production risk (that trade-off in more depth).
• Correctness under failures depends on the dedup key and tie-breaker being FULLY deterministic. If the tie-breaker itself is non-deterministic (for example, relying on row physical order rather than an explicit column), a retry after a partial failure can pick a DIFFERENT duplicate copy to keep than the original attempt did, which is a subtle correctness bug that only manifests on retries, not on a clean single run, making it easy to miss in testing.
• Common mistake: deduplicating on a composite of MANY columns ("the whole row minus timestamp") instead of a true business key, which silently fails to catch duplicates that differ in any column at all (for example, a duplicate event re-sent with a corrected but different payload value is not a "duplicate" by a whole-row comparison, even though it represents the same underlying event and should be resolved by the same explicit keep-policy, not accidentally kept as two separate rows).

Direct answer

An organization-wide data-quality program needs to cover people (roles like data stewards with clear accountability), process (policies, standards, and an incident-response playbook), and technology (a chosen tooling stack), rolled out in phases (pilot, scale, sustain) with measurable milestones, since attempting a company-wide rollout all at once, without a proven pilot, is the most common way this kind of initiative fails.

Structured elaboration

• Pilot phase (roughly months 1-2): select one or two high-visibility, high-pain pipelines as the pilot, implement the full program (stewardship, standards, tooling) end-to-end for just those, and use the pilot to prove the model works and generate a concrete before/after story to build momentum with.
• Scale phase (roughly months 3-4): expand from the pilot to the next tier of critical pipelines, using lessons learned (what tooling choices worked, what process friction showed up) to refine the approach before it becomes standard practice everywhere.
• Sustain phase (roughly months 5-6 and beyond): make the standards and tooling the default for new pipelines going forward, establish ongoing training and onboarding for new team members, and define success KPIs that are tracked continuously rather than only during the initial rollout.
• Tooling choices: the technology leg is not just "pick a tool," it is picking the tool whose friction matches how the organization already works. For a team already heavily invested in dbt-based SQL transformations, dbt tests (the built-in and dbt-native unique, not_null, accepted_values, relationships tests, plus custom SQL-based tests) are the lower-friction choice: they live alongside the existing models, run in the same dbt test invocation the team already uses, and require no new language or deployment surface. For an organization needing richer, cross-language validation (Python-based pipelines, non-dbt sources, or a need for a structured, queryable validation report rather than a pass/fail test run), Great Expectations is the better fit: it is a standalone Python framework that produces a structured "data docs" report and integrates with a wider range of data sources than a SQL-only tool can. The right call is decided by which cost the team can least afford, the friction of adopting a new tool outside their existing stack, or the limitation of a SQL-only tool when the pipeline is not SQL-only, not by which tool is abstractly "better."
• Driving adoption across teams that do not report to you: this is fundamentally an influence problem, not an authority one; the pilot's concrete, quantified success story (a real incident prevented, a real cost saved) is usually the most effective adoption lever, more so than a mandate from above, because it gives other team leads a self-interested reason to opt in rather than a compliance obligation to resent.

Worked example

A six-month program: months 1-2 pilot the full program on the two pipelines that generated the most incidents last quarter, producing a concrete result (for example, a measured 80% reduction in data-quality-related incident tickets for those two pipelines); months 3-4 use that result to recruit three more team leads to adopt the same standards and tooling voluntarily, refining the onboarding process based on friction the pilot teams reported; months 5-6 make the validated tooling the default for any new pipeline going forward and establish a recurring quarterly review of the program's KPIs (incident rate, time-to-detection, percentage of critical datasets with defined quality checks) to the leadership team.

Trade-offs and pitfalls

The failure mode this phased approach specifically avoids is a mandated, all-at-once rollout with no proven pilot: teams asked to adopt an unproven process purely by directive tend to comply minimally and revert as soon as attention moves elsewhere, whereas teams who see a peer team's concrete, quantified success story are far more likely to adopt genuinely and sustain it. The trade-off is speed, a pilot-first approach is slower to reach full coverage than a mandate, but it is far more likely to actually stick once it gets there.

Direct answer: Filter to failed attempts, group consecutive failures for each user into a run using a running count of successes (each success starts a new run of failures), number the failures within each run, and then check whether the timestamp 3 positions apart in the same run is within 10 minutes of the first. This avoids both false positives (isolated failures with successes or long gaps between them break the run before they can combine into 3) and duplicate alerts, because it flags a single deterministic window_start per qualifying triple rather than every possible sliding window over the same failures.

Structured elaboration

1. Group consecutive failures. A running SUM of a success indicator (1 for success, 0 for failure), ordered by time, increments only when a login succeeds. Between two successes, that running total is constant, so it acts as a group id: every failure that happens between the same pair of successes (or before the first success) shares one group id. This is the identical gaps-and-islands mechanism used to find consecutive-day streaks elsewhere in this topic, applied here to failure runs instead of calendar days.
2. Number failures within each run. ROW_NUMBER() partitioned by (user, group id) gives each failure its position in its run: 1st, 2nd, 3rd failure since the last success.
3. Check the 10-minute window on a triple. Self-join the numbered failures to themselves, matching row N to row N+2 in the same run, and keep pairs where the timestamp gap is at most 10 minutes. The earlier row's timestamp is the offending sequence's window_start.

WITH ordered AS (
  SELECT user_id, attempted_at, success,
    SUM(CASE WHEN success THEN 1 ELSE 0 END) OVER (
      PARTITION BY user_id ORDER BY attempted_at ROWS UNBOUNDED PRECEDING
    ) AS success_grp
  FROM logins
),
fails AS (
  SELECT user_id, attempted_at, success_grp,
    ROW_NUMBER() OVER (PARTITION BY user_id, success_grp ORDER BY attempted_at) AS fail_idx
  FROM ordered
  WHERE success = FALSE
)
SELECT DISTINCT n1.user_id, n1.attempted_at AS window_start
FROM fails n1
JOIN fails n3
  ON n1.user_id = n3.user_id
 AND n1.success_grp = n3.success_grp
 AND n3.fail_idx = n1.fail_idx + 2
 AND n3.attempted_at - n1.attempted_at <= INTERVAL '10 minutes'
ORDER BY user_id, window_start;

Worked example (executed in DuckDB). User 1: 3 failures at 09:00, 09:02, 09:04 (all within 4 minutes, then a success), plus an isolated failure at 09:30. User 2: 3 failures at 10:00, 10:20, 10:40 (20 minutes apart each, no success in between, so all one run, but spanning too much time).

flagged_user_id | window_start
1                | 09:00:00

Only user 1 is flagged. User 2's three failures are genuinely consecutive (same run, no intervening success) but span 40 minutes end to end, so the 3rd-failure-minus-1st-failure gap (40 minutes) exceeds 10 minutes and correctly does not trigger.

How this avoids false positives. An isolated failure surrounded by successes, or spread far from other failures in time, never gets two more failures within the same success_grp inside a 10-minute span, so it never satisfies the fail_idx = fail_idx + 2 self-join condition. The grouping by success_grp is what prevents counting a failure from a completely different, much earlier attack attempt toward the current count: any success in between resets the run.

How this avoids duplicate alerts for overlapping sequences. A naive rolling-window formulation (check every failure against every failure exactly 10 minutes later) would fire once per qualifying failure inside a longer run of 4+ failures, producing overlapping, redundant alerts for what is really one incident. This query instead reports exactly one window_start per distinct triple (fail_idx, fail_idx+2): a run of exactly 3 failures produces exactly one alert; a run of 5 rapid failures produces up to 3 overlapping-but-distinct triples (1-3, 2-4, 3-5), which is a deliberate design choice (each triple is separately true), not a bug, but if the requirement is 'one alert per incident, not one per triple,' take MIN(window_start) per (user_id, success_grp) after this query to collapse a whole run down to a single alert.

Trade-offs & pitfalls

• This detects 3+ consecutive failures within a 10-minute span of THAT triple; it does not detect, say, 2 failures 8 minutes apart followed by a 3rd failure 15 minutes after the first two but still within 10 minutes of the 2nd. If the intended semantics is 'any 3 failures within a rolling 10-minute window regardless of adjacency,' that's a genuinely different, more expensive query (a self-join or a true sliding window count, not a fixed offset of 2).
• Collapsing a long run of failures down to one alert (via MIN(window_start) GROUP BY user_id, success_grp) is a deliberate design decision, not automatic; decide upfront whether security wants one alert per run or one per qualifying triple.
• The self-join is the expensive part at scale: it's proportional to the number of failure pairs 2 apart within a run, which is cheap for normal traffic but can grow for a user under sustained attack with thousands of failures in one run; a streaming engine with a native session/window operator is usually a better fit for production-scale abuse detection than a batch self-join.

Direct answer
Write a short summary right after the meeting that states the decision made, names an owner and deadline for each follow-up, and flags anything still unresolved, so nobody has to reconstruct what happened from memory a week later.

Structured elaboration

1. State the decision first, in one sentence, even if it feels obvious right after the meeting; it stops being obvious within a day or two, especially for people who weren't in the room.
2. List action items with an owner and a deadline each, not a bare to-do list; "someone should look into X" is not actionable, "Priya will check the vendor SLA by Thursday" is.
3. Name what's still open, explicitly, rather than letting it quietly drop; a one-line "not yet decided: whether we notify customers proactively" prevents someone assuming it was implicitly settled.
4. Send it promptly, ideally within the hour, while the details are fresh and before people have moved on to something else and stopped tracking it mentally.
5. Keep it short. Three to six sentences is usually enough; a summary that's as long as a transcript won't get read.

Worked example
"Decision: we're moving the schema migration to next Tuesday's low-traffic window instead of doing it live this week. Action items: Priya to update the migration runbook by Monday EOD; Sam to notify the on-call rotation of the new window by Friday. Open question: whether we need a customer-facing heads-up, still deciding, will confirm by Wednesday."

Three sentences, one decision, two owned action items with deadlines, and one explicitly flagged open item.

Trade-offs and pitfalls

• The most common failure is writing a summary that lists what was discussed instead of what was decided; a meeting can generate a page of discussion and one real decision, and the summary should reflect that ratio.
• An action item without a named owner tends to silently not get done; if you can't name an owner in the summary, that's a sign the meeting didn't actually resolve who's responsible.
• Sending it too late (days later) defeats the purpose; by then people have already formed their own, sometimes conflicting, memory of what was agreed.

Direct answer

Store the raw event timestamp in UTC on the fact table (never convert at write time), and store the local business date/time as a SEPARATE derived attribute computed using the relevant time zone at query or extract, transform, load (ETL) time, keyed off the dimension that determines which time zone applies (the user's or store's location). Never overwrite or discard the original UTC timestamp.

Structured elaboration

• Why keep UTC as the source of truth: local time is a function of UTC time plus a time zone, and time zones (and their daylight-saving offsets) change; if you only store a converted local time and discard the UTC original, you lose the ability to correctly re-derive local time later if the zone or DST rules change, and you can't reliably compare events across different local time zones at all.
• Deriving local business time: join the fact (with its UTC timestamp) to a time-zone lookup (either on the relevant dimension, like store_dim.timezone, or a separate country_timezone table) and compute local time at query or ETL time: event_time_utc AT TIME ZONE store_dim.timezone. This correctly accounts for daylight-saving transitions if the underlying time zone conversion function is DST-aware (most modern SQL engines and time zone libraries are).
• Avoiding double-counting across date boundaries: aggregating "daily" metrics must group by LOCAL date (derived from local time), not the UTC date, or a single business day's activity gets split across two UTC dates near midnight local time, undercounting or overcounting depending on the time zone offset direction. Precompute a local_date_key attribute per event during ETL if this grouping happens frequently, to avoid repeating the conversion in every query.
• Materializing versus computing on demand: for high query volume against this pattern, materialize the local date/time as its own column at load time (computed once) rather than repeating the timezone conversion in every downstream query, which is both a performance optimization and a consistency guarantee (every query using the same precomputed local date, rather than each one potentially implementing the conversion slightly differently).

Worked example

An event occurs at 2026-03-08 06:30:00 UTC. For a user in New York (UTC-5 in early March, before the typical spring-forward DST transition), the local time is 2026-03-08 01:30:00, still March 8th locally. For a user in Tokyo (UTC+9), the same UTC timestamp is 2026-03-08 15:30:00, also March 8th locally, but a different clock time. If instead the event happened at 2026-03-08 03:00:00 UTC, the New York local time (UTC-5) is 2026-03-07 22:00:00, a DIFFERENT calendar date locally than the UTC date, exactly the kind of boundary case that breaks naive UTC-date-based daily aggregation for that user.

Trade-offs and pitfalls

The classic bug is grouping "daily active users" or similar metrics by the UTC date column directly, which systematically misattributes events near midnight to the wrong LOCAL day for every time zone except UTC itself, and does so silently (the query runs fine, the numbers just don't match what a local observer would report). Precomputing and testing the local-date derivation against known DST transition dates for major time zones is worth the ETL investment for any dashboard reporting daily metrics to a geographically distributed audience.

Direct answer. Default per-column statistics assume each column's values are distributed independently of every other column, so when two columns are actually correlated, the optimizer multiplies their individual selectivities together and ends up with an estimate far lower (often wildly so) than the true combined selectivity; extended statistics explicitly capture how those columns co-vary, letting the optimizer estimate the combined predicate's selectivity directly rather than assuming independence.

Structured elaboration. If a city column and a state column are each individually somewhat selective on their own, but every value of city in the data only ever co-occurs with one specific state (a real-world correlation: cities belong to exactly one state), the independence assumption badly overestimates how selective city = X AND state = Y really is: multiplying each column's standalone selectivity together implies the combined predicate is far MORE selective than it actually is, since knowing the city already tells you the state with certainty. Extended (multi-column) statistics record the actual joint distribution across a specified set of correlated columns, so the optimizer can look up (or closely approximate) the real combined selectivity instead of computing a product that assumes independence.

Worked example. Two predicates might each independently match 10% of rows on their own; if they were truly independent, the combined predicate would match roughly 1% (10% times 10%). If the columns are perfectly correlated (every row matching one predicate also matches the other), the true combined match rate is still 10%, not 1%, a ten-fold estimation error that can easily push the optimizer toward an index-heavy plan appropriate for a genuinely rare combination, when the real combination is common enough that a scan-based plan would have been the better choice.

Trade-offs and pitfalls. Extended statistics have to be explicitly created and targeted at the specific column combination that's actually correlated, most engines don't infer correlation automatically and build multi-column statistics for every possible column pair on their own, since that would be prohibitively expensive to maintain; you generally need to notice the estimation error first (via an estimate-actual mismatch on a predicate involving those columns) before you know which specific column combination is worth the investment. Once created, verify the fix by re-checking that predicate's estimated-vs-actual row count, rather than assuming creating the statistics automatically resolved the issue, since the statistics also need to be genuinely representative of the current data to help.

Situation: At my last company we needed to reduce time-to-insight for business analysts on user behavior – PM goal was “enable analysts to run product funnels within 2 hours of event occurrence” to improve experimentation speed.

Task: As the data engineer owner, I needed to explain why this mattered to PMs (business impact) and to engineers (technical scope) and produce artifacts that aligned both groups.

Action:

• One-pager for PMs: problem statement, business goal, target metric (median freshness ≤ 2 hours), constraints (cost budget, retention policy), and trade-offs (near-real-time vs. backfill complexity). This used simple graphs showing current latency distribution and business impact (e.g., delayed experiments).
• Kickoff slides for engineers: architecture sketch (stream ingestion → processing → warehouse), throughput/latency requirements, expected SLA, error budget, and required changes (add Kafka topic, Spark Structured Streaming job, incremental CDC).
• PRD section “Acceptance Criteria” (clear, testable):
  1. Event pipeline processes ≥ 99% of events within 2 hours end-to-end.
  2. Mean event lag ≤ 30 minutes; 95th percentile ≤ 2 hours.
  3. Data schema versioning in place; no breaking changes without migration.
  4. Monitoring: alerts on lag > 2 hours for >5 minutes and daily data quality report.
• Implementation tasks mapped to sprint tickets with definitions of done (unit tests, integration test against sample traffic, runbook).

Result: PMs accepted the trade-offs; engineers had unambiguous targets and built the streaming pipeline. Within 6 weeks median freshness dropped to 15 minutes and 95th percentile to 1.3 hours; experiment iteration time halved.

What I learned: Always state the business outcome first, then translate it into measurable SLAs and concrete acceptance criteria. Use different artifacts (one-pager for PMs, technical kickoff and PRD acceptance criteria for engineers) so each audience gets the "why" framed in terms they act on.

A staging (or landing) area is a raw, largely untransformed copy of source data that sits between extraction and the curated tables analysts actually query. You keep it separate for three reasons that all come back to the same idea: the raw copy is your safety net.

What lives there and what happens to it

• Data lands close to its source shape (same columns, minimal type coercion) so a transformation bug never destroys information you didn't capture anywhere else.
• Light operations happen here before anything moves further downstream: basic type casting, deduplication of exact source-level duplicates, and enrichment that has to happen once (attaching a load timestamp, a source system tag, a batch id).
• It is the recovery point. If a downstream transform is wrong, you re-derive the curated table from staging instead of re-extracting from the source system, which may be slow, rate-limited, or (for a point-in-time correction) no longer possible to reproduce exactly.

Retention
Staging data is usually cheap: raw storage, no indexes, no BI-facing service-level agreements (SLAs) to honor, so a common policy is to keep it far longer than the curated layer needs, often 30 to 90 days on a rolling window, sometimes indefinitely for regulated or audit-sensitive domains. The retention call is really a bet: how far back would you ever need to reprocess from raw, versus what the storage costs to keep that option open.

Trade-offs and pitfalls
Skipping staging (transforming directly from source into curated tables) is tempting because it looks like fewer moving parts, but it collapses your only recovery path into "re-run the extraction," which does not always give you the same data twice (source systems get pruned, APIs paginate differently over time, upstream tables get purged). The opposite failure is treating staging as query-able and letting analysts hit it directly: it has none of the cleaning, deduplication, or documentation the curated layer promises, so ad-hoc use of staging tends to produce numbers that quietly disagree with the official dashboard.

Direct answer

Move to a dedicated warehouse when ad-hoc analytical queries start measurably hurting the production database's transactional performance, when the same numbers are being computed slightly differently in different spreadsheets, or when reporting needs data joined across sources the production database does not have (a payments processor, a support tool, a marketing platform). Start with the simplest useful version: a small set of tables that are periodically copied out of production into a separate database or a managed cloud warehouse, denormalized just enough to answer the handful of reports people actually run today, not a fully modeled Kimball bus architecture with conformed dimensions across every future business process.

Structured elaboration

Signal one: production impact. A heavy analytical query (a full table scan for a monthly report, say) run directly against the database serving live user traffic can degrade transactional latency for real users; if analysts are being asked to "only run reports at night" or engineers are seeing production incidents traced to a report someone ran, that is a concrete, observable signal, not a vague sense that things feel slow.

Signal two: inconsistent numbers. Once more than one person is computing the same metric independently (one analyst's spreadsheet formula, another's ad-hoc query), small differences in filtering or date handling silently produce different answers to "what was our revenue last month," and nobody notices until two answers are compared in the same meeting. This is the earliest, cheapest form of the exact conformance problem later covered by dimension conflicts across marts; catching it before it compounds is far cheaper than the reconciliation project.

Signal three: joining across sources. Once a report needs to combine production order data with a separate support tool's ticket data and a third-party payment processor's transaction data, there is no single production database to query against anymore; some place has to receive copies of all three and let them be joined together, which is the core job a warehouse exists to do.

The simplest version to build first. Do not start with a full dimensional model. Start with a small, straightforward extract-and-load process (even a scheduled job that copies a handful of production tables into a separate database or a managed cloud warehouse on a nightly cadence) and let analysts query those copies directly, denormalized or lightly modeled, for exactly the reports people already run. Introduce actual dimensional modeling (declared grain, a real date dimension, slowly-changing-dimension handling) only once a second or third report reveals that ungoverned ad-hoc structure is producing inconsistent answers or is too slow to maintain by hand, which is the point at which the methodology and system-design questions the rest of this topic covers actually become relevant.

Worked example

A single unindexed analytical query scanning a 10-million-row production orders table for a monthly report can hold a lock or consume enough I/O bandwidth to add hundreds of milliseconds to unrelated transactional queries hitting the same table concurrently; at a company processing customer-facing checkout requests against that same table, a delay large enough for customers to notice during checkout is the concrete, business-visible cost of skipping a warehouse, not an abstract inefficiency. That single observation, "a report degraded checkout latency," is usually the moment a 20-person startup's engineering leadership actually approves the investment, well before any of the modeling-methodology questions in this topic become the operative concern.

Trade-offs and pitfalls

The most common mistake at this stage is over-building: reaching for a full Kimball-style bus architecture, multiple conformed dimensions, and Type 2 slowly changing dimension (SCD) history tracking before there is more than one or two reports that need any of it wastes engineering effort the startup does not have to spare, and most of that early investment will be redesigned anyway once real reporting needs are better understood. The opposite mistake, waiting until the production database is visibly struggling before doing anything, is also common and more expensive to unwind, since by then inconsistent numbers have usually already reached several audiences and eroded trust in whichever spreadsheet or dashboard people were relying on.

Direct answer
A good 30-60-90 plan moves someone from learning the environment, to contributing under supervision, to owning outcomes independently, with the phase boundaries defined by demonstrated behavior (what they can do unsupervised) rather than by the calendar alone. Track it with a small number of concrete, visible outputs per phase so "on track" is something you can point to, not just a feeling.

The three phases, by what changes

• Days 1-30 (learn and observe): environment setup, codebase or domain orientation, shadowing, and one small real contribution rather than a toy task, so the first change is real but low-risk.
• Days 31-60 (contribute under guidance): own a medium-sized piece of work end to end with a mentor available for review and unblocking, not doing it alongside them line by line.
• Days 61-90 (own outcomes): lead something (a project, an on-call rotation, a smaller onboarding task for the next hire) with the mentor as a backstop, not a co-pilot.

How you know they're on track

• Define the signal per phase in advance, not retroactively: for phase 1, did they reproduce the environment and ship one small real change without major help; for phase 2, is their review feedback shrinking in volume and severity over successive changes; for phase 3, can they make a reasonable decision alone and only escalate the genuinely hard calls.
• Check in on cadence (weekly early on, less frequent later) rather than waiting for day 30, 60, or 90 to find out something drifted three weeks ago.

Adjusting the plan for real constraints

• Limited training resources: when there's no dedicated ramp-up bandwidth (no spare mentor hours, no formal training material), lean harder on asynchronous artifacts: written runbooks, recorded walkthroughs, a curated list of the most representative recent changes, and a lighter-touch weekly sync instead of daily pairing. The phases stay the same; what changes is how much is self-serve versus live.
• Cross-skill ramp: if someone hired primarily for one skill set is expected to also ship in an adjacent one by day 90 (for example, a backend-focused hire expected to ship frontend work), that adjacent skill needs its own explicit milestone inside the plan, not an assumption it'll happen by osmosis. Concretely: days 1-30 stays focused on their strong area to build early confidence and trust; days 31-60 introduces the adjacent skill on a small, well-scoped, low-risk piece with close review; days 61-90 has them own something end to end in the new area, even if smaller in scope than their core-skill ownership.

Worked example
For a new hire joining an established codebase with a small team and no dedicated onboarding budget (the limited-resources case), the 30-60-90 looked like: days 1-30, self-serve environment setup using a written runbook plus a single half-day pairing session, culminating in one small, real bug fix; days 31-60, ownership of one medium feature with async review as the main touchpoint, and a short weekly 15-minute sync instead of daily check-ins; days 61-90, the new hire wrote the onboarding runbook update for the next person, which served double duty as both a real deliverable and a check on whether they actually understood the system well enough to explain it. Being on track was tracked by a short checklist per phase (environment reproducible, first fix merged with normal review effort, feature shipped with review comments trending down) rather than a single blanket "how's it going" check-in.

Trade-offs and pitfalls

• Treating the day boundaries as fixed calendar dates rather than behavioral milestones creates false confidence; someone can hit day 60 without actually being ready for phase-3 ownership, and pushing them into it anyway sets them up to fail.
• Under-supporting the adjacent-skill ramp (assuming a backend engineer will "pick up" frontend without an explicit milestone) is a common way cross-skill onboarding quietly fails; it needs the same structure as the primary skill, just smaller in scope.
• Compressing the plan under limited training resources by cutting phase 1 short (rushing into real ownership before the environment and codebase are understood) trades a faster-looking ramp for more review overhead and rework later.