Lyft Data Scientist (Entry Level) - Comprehensive Interview Preparation Guide

Core services
I would build a platform with a job API, scheduler, quota service, checkpoint store, metadata catalog, and node agents. The scheduler matches jobs to GPU pools, while the quota service enforces team budgets and priorities.

How jobs get compute
A team submits a job spec with requested GPUs, runtime, and a checkpoint policy. The scheduler grants a lease, which is a time-bounded claim on nodes. Workers heartbeat to renew the lease. On completion, failure, or preemption, the lease ends and the GPUs are released automatically.

Controls

• Per-team quotas and project budgets
• Priority classes for urgent training
• Spot first scheduling with fallback to on-demand
• Mandatory checkpoints for long jobs
• Admission control when the cluster is full

Worked example
A 64-GPU run might get 8 workers with 8 GPUs each and a 30-minute renewable lease. If a node fails, the job restarts from the last checkpoint instead of starting over.

This design keeps cost visible, failures recoverable, and shared capacity fair.

Direct answer

The intercept is the expected outcome for an observation at the reference (baseline) level of every categorical variable and zero on all continuous predictors; each dummy coefficient is the average shift relative to that baseline, holding everything else fixed. You avoid the dummy-variable trap by dropping one level per categorical variable, encoding a K-level category with K-1 dummy columns so the design matrix stays full rank.

Structured elaboration

Encoding: for a K-level categorical variable, create K-1 dummy columns and drop one level (the reference).

Intercept: if continuous predictors aren't centered, the intercept is technically the prediction at continuous = 0, which may be outside the realistic data range. Centering continuous predictors at their sample mean makes the intercept read as "typical prediction for the baseline category" instead of a mathematical extrapolation.

Dummy coefficients: each one is interpreted as "holding the continuous predictors fixed, this category's prediction is [coefficient] higher or lower than the baseline category's prediction."

The dummy-variable trap: including all K dummy columns plus an intercept makes the K dummy columns sum to the constant intercept column, a perfect linear dependency. This makes X^T X singular, so OLS either has no unique solution or the solver silently drops a column / produces unstable coefficients depending on the library. Fixes: drop one level (reference coding, the standard choice), or drop the intercept and keep all K dummies (each coefficient then directly estimates that group's mean), or apply regularization if you specifically want all levels represented (ridge tolerates the redundancy but the individual coefficients lose the clean "versus baseline" reading).

Worked example

Predictors: sqft (continuous) and a 3-level region variable (North = baseline, South, West). Suppose the fitted model is (illustrative coefficients, in $000s):

$\widehat{\text{price}} = 50 + 0.20\,\text{sqft} + 20\,\text{South} + 35\,\text{West}$

The intercept, 50, is the predicted price for a North home at sqft = 0 (a mathematical anchor, not a realistic home). For a South home at sqft = 1500: 50 + 0.20(1500) + 20 = 50 + 300 + 20 = 370 ($000). For a West home at the same sqft: 50 + 300 + 35 = 385 ($000). So, holding square footage fixed, a West home is predicted $15,000 more than an equivalent South home (385 - 370) and $35,000 more than an equivalent North home, which is exactly what the West coefficient of 35 says on its own.

Trade-offs & pitfalls

Forgetting to drop a level triggers the trap; whether that fails loudly or silently depends on the tool (pandas get_dummies keeps all levels unless drop_first=True is passed explicitly; scikit-learn's OneHotEncoder needs drop='first'). The choice of reference level is statistically arbitrary but affects readability, pick a meaningful baseline (most common category, or a natural "control" group) for stakeholder-facing reports. Rare categories can have dummy coefficients with very large standard errors since few observations inform them; grouping rare levels into an "other" bucket is a common practical fix.

Direct answer
A retention curve is best for showing the OVERALL shape and rate of decay for one or a few cohorts at a glance, a cohort heatmap (matrix) is best for scanning patterns across MANY cohorts simultaneously (spotting which cohorts are better or worse, and whether retention is improving over time), and a raw cohort table is best when exact numbers matter more than pattern-spotting, such as for an audit or a precise calculation.

Structured elaboration

• Retention curve: a line per cohort showing retention percentage over days/weeks since acquisition; strongest insight is the SHAPE of decay (does it flatten into a stable "core" retention level, or keep declining); works well for comparing a small number of cohorts (2-5) but becomes cluttered with many overlapping lines beyond that.
• Cohort heatmap (matrix): rows are cohorts (by acquisition period), columns are periods since acquisition, cells colored by retention rate; strongest insight is spotting PATTERNS across many cohorts at once (e.g. a visible "vertical stripe" of low retention across all cohorts in one specific calendar period signals an external/platform-wide issue, not a cohort-specific one); best for analysts scanning for anomalies across dozens of cohorts.
• Raw cohort table: exact numeric retention values per cohort and period; best when a decision genuinely needs the precise number (e.g. computing a specific cohort's exact day-30 retention for a report), or as an underlying export behind either visual chart.
• Executive vs. analyst preference: executives generally prefer the retention curve (a fast "is retention healthy and stable" read for 1-2 headline cohorts) or a single summary number; analysts generally prefer the heatmap (pattern-scanning across many cohorts) supplemented by the raw table for exact figures when needed.

Worked example
A cohort heatmap immediately reveals that every cohort acquired in a specific month shows unusually low week-1 retention, a pattern a single retention curve (showing only one or two cohorts) would never surface, prompting an investigation into what changed platform-wide that month (a UI change, an onboarding bug) rather than assuming that cohort was simply lower quality.

Trade-offs and pitfalls
Using a retention curve to compare many (10+) cohorts produces an unreadable tangle of overlapping lines; that's precisely the situation where a heatmap's color-based pattern-scanning outperforms line-based comparison.

Direct answer

The underlying finding stays identical, but you change altitude, vocabulary, and level of supporting detail. An executive gets the headline, the business impact, and the recommended decision in one or two lines up front. The implementation team gets the mechanism, the caveats, and enough of the underlying data to act on it correctly.

Structured elaboration

• Altitude: conclusion-first for the executive, versus enough method detail for the team to trust and reproduce the diagnosis.
• Vocabulary: business-impact language (revenue, risk, timeline) for the executive, technical specifics (segments, funnels, thresholds) for the team.
• Format: a one-slide or one-paragraph summary versus a working document with a data appendix.
• What must never change: the number itself and the direction of the conclusion, in both versions.

Worked example

Finding: onboarding drop-off at step 3 is costing an estimated 6% of new signups per month.
Executive version: "we're losing about 6 of every 100 new signups at the step-3 confirmation screen, fixing it could recover meaningful revenue this quarter, recommend prioritizing it."
Team version: "62% of that drop-off happens on mobile between form submit and confirmation render, median time to abandon is 9 seconds, this looks like a loading-state issue on mobile specifically."
Both versions agree on the 6% headline number and the recommendation to prioritize the fix.

Trade-offs and pitfalls

The two versions can quietly drift into different conclusions if you're not careful, always trace both back to the same underlying analysis. Over-simplifying for the executive can also strip out the one caveat that would have changed their decision, so pick what to omit deliberately, not by default.

What the interviewer probes next

Expect a question about what happens when the executive summary gets forwarded on without you in the room, and how you prevent it from being read out of context.

Direct answer

For a single numeric column, start with a histogram (or a KDE, a kernel density estimate) for shape, a boxplot for a compact five-number summary and outlier flags, and the standard descriptive statistics (mean, median, standard deviation, min, max). For a single categorical column, start with a bar chart of value counts and the distinct-value count itself, since cardinality changes what "summarizing" even means.

What each reveals that the others don't

A histogram shows the full shape: modality, skew, and gaps that a single summary number can't convey. A boxplot compresses that shape into quartiles and flags candidate outliers, which is more compact but hides multimodality (two distinct humps can produce a perfectly ordinary-looking boxplot). Descriptive statistics give you exact numbers to reference or compare across columns, but on their own they can be actively misleading on a skewed or multimodal distribution (the classic case: a dataset can have identical mean, variance, and correlation as another while looking completely different when plotted). For categorical data, a bar chart of value counts shows both the shape (is one category dominant, or is it fairly even) and immediately reveals cardinality problems (hundreds of near-unique values where you expected a handful of categories).

Worked example

For a price column: the histogram shows a long right tail with most values clustered under $50 and a scattering up past $500; the boxplot confirms a compact box around $20 to $45 with several points flagged above the upper fence; and the mean ($68) sitting noticeably above the median ($38) confirms the skew numerically. For a payment_method column: value counts show credit_card at 71%, paypal at 24%, and four other methods splitting the remaining 5%, with a distinct count of 6, telling you immediately this is a genuinely low-cardinality categorical column you can treat straightforwardly.

Trade-offs and pitfalls

Don't stop at descriptive statistics alone for a column you haven't looked at before: they're fast to compute but can hide exactly the shape information (multimodality, heavy tails, a spike of a single repeated value) that a chart would reveal in seconds. For categorical columns, always check the distinct count before choosing a visualization, since a bar chart of value counts is useless once cardinality climbs into the hundreds or thousands.

Direct answer

Keep alignment across time zones with three levers: shrink what actually needs real-time overlap by defaulting to async updates on a fixed template, protect a small deliberately scheduled overlap window for anything that truly needs live discussion, and make handoffs explicit in writing so context transfers cleanly across the boundary instead of depending on someone's memory.

Framework

Reduce dependence on overlap. Default to async status updates on a fixed cadence, and use written decision docs rather than requiring a live meeting for every decision. Most updates don't need a room, only genuinely ambiguous or high-stakes calls do.

Protect a deliberate overlap window. Negotiate a recurring block, even a short one, and rotate who takes the inconvenient time so the burden doesn't always fall on the same region.

Make handoffs explicit. When work crosses a time-zone boundary, produce a short written artifact rather than relying on a quick chat message. This matters most in ops-heavy, always-on contexts.

Worked example

Consider an on-call rotation providing 24/7 production coverage across three time zones (for example [Region A], [Region B], and [Region C]), where the two outer regions have little or no live overlap with each other.

• Shadow and overlap periods: the incoming region's on-call shadows the outgoing region's on-call for a short deliberate window at the shift boundary, even 15 to 30 minutes, to ask questions live before the outgoing engineer signs off.
• Written handoff template: a standard document filled at every handoff covering open incidents, any systems in a degraded state, changes deployed in the last shift, and explicit 'known risk' or 'do not touch' notes.
• Escalation expectations: a written policy defining what counts as page-worthy versus a handoff note, who the secondary on-call is in each region, and how long the incoming engineer has to acknowledge before it auto-escalates.
  Result: even with zero live overlap between two of the three regions, the written handoff plus the short shadow window from the middle region means each incoming on-call starts already briefed, instead of reconstructing state from raw logs.
  For non-ops roles the same mechanism applies with a different artifact, for example a design or product handoff might be a written decision log plus a recorded walkthrough rather than an incident handoff, but the principle (explicit written handoff over a live conversation) is the same.

Trade-offs and pitfalls

• Repeatedly scheduling occasional syncs at painful hours burns out whichever time zone draws the short straw. Rotate it deliberately.
• Async-only breaks down for genuinely ambiguous or high-stakes decisions. Some live channel for true emergencies still has to exist.
• A handoff template that's too heavy gets skipped under time pressure. Keep it short enough to fill in within a few minutes.
• Assuming a chat message counts as a handoff is the actual failure mode this whole approach is designed to prevent. The structured artifact is the point, not the tool it's written in.

Direct answer

Split the logic into a fit step and a transform step: fit computes and stores a median per group (only for groups with enough non-missing observations) plus one overall median as the fallback, and transform looks up each row's group median, falls back to the overall median when the group is unseen, too small, or was entirely missing, and fills only the rows where the target is actually NaN (not-a-number, pandas' missing-value marker). Storing the fitted medians as plain attributes means the exact same transform logic can be reapplied to new data without recomputing anything from the original training set.

Structured elaboration

• Fit: group by the region column, compute the median of the target column per group (pandas' median() already ignores NaN within each group), and separately count non-null observations per group.
• Reliability threshold: a group's stored median is only trusted if it had at least min_group_size non-missing observations at fit time. Groups below that threshold, or groups that never appeared at fit time at all, fall back to the single overall median. This directly matches "too few observations, or was never seen at all" from the question, both routes land on the same fallback.
• Transform: map each row's group to its stored median (or NaN if the group wasn't kept), fill the gaps in that mapped Series with the global median, then use it only to fill rows where the target is actually missing, leaving already-present values untouched.
• Packaging this as a small class with fit/transform methods (the scikit-learn estimator convention) is what makes "fit once, reapply consistently" concrete: the fitted medians live on the object, not recomputed inline, so new data goes through transform alone and gets exactly the same group-to-median mapping the training data used.

Worked example

import pandas as pd

class GroupMedianImputer:
    def __init__(self, group_col, target_col, min_group_size=3):
        self.group_col = group_col
        self.target_col = target_col
        self.min_group_size = min_group_size

    def fit(self, X):
        non_null = X.dropna(subset=[self.target_col])
        counts = non_null.groupby(self.group_col)[self.target_col].size()
        medians = non_null.groupby(self.group_col)[self.target_col].median()
        # only trust a group's median if it had enough observed values
        reliable = medians[counts >= self.min_group_size]
        self.group_medians_ = reliable.to_dict()
        self.global_median_ = X[self.target_col].median()
        return self

    def transform(self, X):
        X_out = X.copy()
        fill_values = X_out[self.group_col].map(self.group_medians_).fillna(self.global_median_)
        mask = X_out[self.target_col].isna()
        X_out.loc[mask, self.target_col] = fill_values[mask].values
        return X_out

# fit on historical data with three regions: two well-observed, one tiny, one all-missing
train_df = pd.DataFrame({
    'region': ['east', 'east', 'east', 'west', 'west', 'west', 'tiny', 'north'],
    'income': [50000, 52000, 48000, 60000, 62000, 64000, 70000, float('nan')],
})
imputer = GroupMedianImputer('region', 'income', min_group_size=3)
imputer.fit(train_df)

# reapply the same fitted logic to new data, including an unseen region
incoming_df = pd.DataFrame({
    'region': ['east', 'west', 'tiny', 'south', 'north'],
    'income': [float('nan')] * 5,
})
new_df = imputer.transform(incoming_df)

Verified: imputer.group_medians_ keeps only east (50000.0) and west (62000.0), the two regions with at least min_group_size=3 non-missing observations; tiny (1 observation) and north (all-missing) are correctly excluded from group_medians_. Running transform on incoming_df fills east and west with their own group medians, and correctly falls back to the overall median (60000.0) for tiny, the never-seen south, and the all-missing north, confirming all three fallback triggers named in the question, too few observations, never seen, and entirely missing, resolve to the same value.

Trade-offs and pitfalls

• Complexity: fit is O(n) for the groupby plus counting pass; transform is O(n) for the map and fill. Memory is O(g) for the stored medians, where g is the number of distinct groups, independent of how large future batches are.
• Edge cases: a group with observations but they're all above/below normal ranges still gets a legitimate median, that's expected, not a bug; the target column being entirely NaN at fit time makes the global median itself NaN, so decide explicitly whether that should raise rather than silently leaving rows unfilled; non-numeric or badly-typed group keys (e.g. a mix of int and str region codes) will map inconsistently, so normalize the group key's dtype before fitting.
• A common mistake is computing the fallback as "global median of the fit set" but then never re-checking it against new data: if the new data's distribution shifts meaningfully from what was fit, the stored medians go stale silently, since nothing here re-fits automatically. This is a reasonable trade for reproducibility (the same input always imputes the same way), but it does mean this needs periodic re-fitting in production, the same way any other fitted preprocessing step does.
• min_group_size is itself a judgment call: too low and a two-observation "median" is really just picking one noisy value; too high and legitimately small-but-real segments always fall back to the (potentially very different) global median. There's no universally correct default; pick it based on how much you trust a small sample's median in this specific dataset.

Direct answer: Recompute the affected one-week period from the corrected historical events, validate the recomputed values against expectations before publishing, version the correction distinctly from the original run, and publish it with a clear audit trail showing what changed and why, minimizing disruption by targeting exactly the affected week rather than a broader reprocessing.

Structured elaboration, step by step:

1. Scope the partial backfill precisely: identify exactly which dates and which metrics are affected by the historical event updates (not a broader window "just to be safe," which wastes compute and, more importantly, needlessly changes numbers that were never actually wrong).
2. Recompute from the corrected source data: re-run the metric pipeline for exactly the affected week, reading the now-updated historical events, producing a new candidate output tagged with a new data_version.
3. Validate before publishing: spot-check the recomputed week's values against the specific records that were updated (do the numbers move in the expected direction and rough magnitude given what changed), and run the standard golden-fixture/reconciliation tests (S18, S36) against the new output before it replaces anything visible.
4. Publish with an audit trail: the published correction records the previous value, the new value, the reason (which upstream event updates drove the correction), and the timestamp of the correction itself, so a consumer noticing a changed historical number can trace exactly why, without needing to ask an engineer.
5. Minimize disruption to downstream consumers: notify known consumers of the affected week's metrics BEFORE the correction goes live wherever feasible (especially anything feeding an OKR, an alert threshold, or another dependent model), and prefer additive/versioned publishing (the old value remains queryable under its version tag) over silently overwriting history with no trace of the prior value.

Worked example: if historical event updates affect the week of May 10-16 specifically, the backfill job targets exactly that week, recomputes DAU/revenue for those 7 days, validates the recomputed numbers move in a direction consistent with what changed in the underlying events (e.g., if the update added previously-missing purchase events, revenue should move UP, not down or unpredictably), and publishes with an audit note: "May 10-16 revenue corrected on 2026-06-05, +2.1%, due to a late-arriving batch of purchase events for that week; see data_version v3 vs v2."

Trade-offs & pitfalls: A common shortcut is reprocessing a WIDER window than strictly necessary "to be thorough," which unnecessarily perturbs numbers that were never wrong and makes it harder for a consumer to understand exactly what changed and why; precise scoping to the genuinely affected period, backed by the audit trail, is what keeps a partial backfill from becoming its own source of confusion.

ROLLUP produces a hierarchy of subtotals in one query: detail rows, then progressively higher-level subtotals, culminating in a grand total, all without running separate GROUP BY queries and UNIONing them together.

Structured elaboration

SELECT date, region, SUM(amount) AS total
FROM sales
GROUP BY ROLLUP(date, region);

ROLLUP(date, region) produces, in one pass: detail rows grouped by (date, region), then subtotal rows grouped by date alone (with region shown as NULL), then a single grand-total row (with both date and region shown as NULL). The NULLs appearing in the grouping columns of a ROLLUP result are not missing data, they specifically mean "this row is a subtotal or total across all values of this column", a signal you can detect and label using GROUPING(). But ROLLUP is a strict, nested hierarchy in the order its arguments are listed: ROLLUP(date, region) only ever collapses region into date-level subtotals, it never independently collapses date to produce a region-only subtotal, because a by-region subtotal isn't one of the levels the date-then-region hierarchy passes through. Since the question asks for subtotals by date AND by region AND a grand total, three genuinely independent subtotal shapes, plain ROLLUP can't produce all three; GROUPING SETS is the tool for that, since it lets you list each grouping combination independently instead of only a nested hierarchy:

SELECT date, region, SUM(amount) AS total
FROM sales
GROUP BY GROUPING SETS ((date, region), (date), (region), ());

Each parenthesized item in GROUPING SETS is its own independent grouping: (date, region) for the detail rows, (date) for date-only subtotals, (region) for region-only subtotals, and () for the grand total, all four computed in one pass and combined into one result.

Worked example

Given sales(date, region, product, amount) with two rows on 2025-01-01 (east: 100, west: 50) and one row on 2025-01-02 (east: 30): ROLLUP(date, region) produces the 3 detail rows, plus a subtotal for 2025-01-01 (150, summing east and west), a subtotal for 2025-01-02 (30), and a grand total row (180, summing everything), 6 rows total, but NO region-only subtotal anywhere in that output: there is no row showing east's total across both dates (130) or west's total (50) on its own.

GROUPING SETS ((date, region), (date), (region), ()) on the same data produces all of that PLUS the missing region subtotals, 8 rows total: the same 3 detail rows; date subtotals 2025-01-01 = 150 and 2025-01-02 = 30; region subtotals east = 130 (100 + 30, across both dates) and west = 50; and the grand total 180. This is the query that actually answers "subtotals by date, by region, and an overall grand total" as the question asks; ROLLUP(date, region) alone silently leaves the by-region half of that ask unmet.

Trade-offs and pitfalls

ROLLUP is hierarchical and order-sensitive: ROLLUP(date, region) rolls up region into date subtotals, not the reverse; ROLLUP(region, date) would instead produce per-region subtotals across all dates. CUBE, by contrast, produces every possible combination of subtotals (all combinations of grouping columns rolled up, not just the nested hierarchy), which is the right tool when you need cross-tabulated totals rather than a strict drill-down hierarchy.

Approach (brief):

• Simulate many trials of sequential A/B tests with K correlated metrics. At each interim look compute test statistics for each metric (e.g., difference-in-means → z-scores using known/pooled SD). Generate correlated metric outcomes per user/experiment arm via multivariate normal with specified correlation matrix and per-metric effect sizes. Use an alpha-spending function (O'Brien–Fleming) to allocate cumulative alpha across L looks, convert to per-look boundaries (two-sided) then apply them to the simulated z-scores. Record first look crossing for any metric (family-wise decision). Aggregate empirical Type I error (when true effects = 0) and power (when effects > 0) across simulations.

Key choices & reasoning:

• Simulate raw outcomes per subject via multivariate normal to respect correlation structure; this properly captures joint distribution of z-statistics.
• Use O'Brien–Fleming spending: cumulative alpha_t = 2*(1 - Phi(z_alpha / sqrt(t_frac^{-1}))) — implement standard formula (or use numeric root to find boundary per look so cumulative spent equals spending function).
• At each look, test family-wise: reject null if any metric's |z| > boundary. Because correlation is simulated directly, FWER is captured empirically without ad-hoc correction.

Code skeleton (Python):

import numpy as np
from scipy.stats import norm, multivariate_normal
from math import sqrt

def obrien_fleming_alpha_spend(alpha, look_fractions):
    # returns two-sided critical z for each look so cumulative alpha follows OF spending
    # simple conservative approximation: boundary z_t = norm.ppf(1 - alpha/(2* (1.0/len(look_fractions))))
    # Better: use repeated numeric search so that cumulative spent at look t equals spending function.
    def spend(t):
        # OF cumulative spending at fraction t (Lan-DeMets OF approx)
        return 2 * (1 - norm.cdf(norm.ppf(1 - alpha/2) / sqrt(t)))
    z_bounds = []
    for t in look_fractions:
        cum_alpha = spend(t)
        z = norm.ppf(1 - cum_alpha/2)
        z_bounds.append(z)
    return np.array(z_bounds)

def simulate_one_trial(n_per_arm_by_look, K, mu_control, mu_treatment, cov_matrix):
    # simulate incremental additional users between looks
    # returns list of z-scores per look (K,) computed from cumulative data
    cumulative_N = 0
    sum_control = np.zeros(K)
    sum_treat = np.zeros(K)
    for add_n in n_per_arm_by_look:
        # simulate add_n new users per arm with multivariate normal metrics
        Xc = multivariate_normal.rvs(mean=mu_control, cov=cov_matrix, size=add_n)
        Xt = multivariate_normal.rvs(mean=mu_treatment, cov=cov_matrix, size=add_n)
        sum_control += Xc.sum(axis=0)
        sum_treat += Xt.sum(axis=0)
        cumulative_N += add_n
        # compute pooled mean and sd per metric
        mean_c = sum_control / (cumulative_N)
        mean_t = sum_treat / (cumulative_N)
        # sample variances (pooled) — using per-subject simulated variance
        # For speed in large sims, approximate se = sqrt(2*diag(cov)/(cumulative_N))
        se = np.sqrt(2 * np.diag(cov_matrix) / cumulative_N)
        z = (mean_t - mean_c) / se
        yield z  # z is array length K for this look

def run_simulation(n_sims, n_per_arm_by_look, K, effect_sizes, corr, alpha=0.05):
    # prepare parameters
    cov = np.outer(np.sqrt(effect_sizes_var := np.ones(K)), np.sqrt(effect_sizes_var)) * corr
    # here assume unit variances for metrics; effect_sizes are shifts in means for treatment
    mu_control = np.zeros(K)
    mu_treatment = np.array(effect_sizes)
    look_fracs = np.cumsum(n_per_arm_by_look) / sum(n_per_arm_by_look)
    z_bounds = obrien_fleming_alpha_spend(alpha, look_fracs)
    rejections = 0
    for sim in range(n_sims):
        for look_idx, z in enumerate(simulate_one_trial(n_per_arm_by_look, K, mu_control, mu_treatment, cov)):
            if np.any(np.abs(z) > z_bounds[look_idx]):
                rejections += 1
                break
    power = rejections / n_sims
    return power

Key points:

• Use multivariate_normal.rvs to simulate correlated metric vectors per subject; this preserves metric correlation.
• Compute z using cumulative sample sizes; se approximated from known covariances (for large-sample A/B tests this is fine). For small samples compute empirical variances from simulated data instead.
• O'Brien–Fleming spending implemented via Lan-DeMets approximation; for exact boundaries perform numeric solving to match cumulative alpha.

Complexity:

• Time: O(n_sims * total_sample_size * K) — dominated by random draws.
• Memory: O(K^2) for covariance; per-sim incremental memory small.

Edge cases and extensions:

• Binary or non-normal metrics: simulate from appropriate joint copula (e.g., Gaussian copula with marginals transformed).
• Unknown variances: estimate variances from data within each simulation.
• Multiplicity rules: current approach evaluates family-wise crossing of any metric; you can instead control per-metric alpha or use simultaneous test statistics with multivariate crossing thresholds derived analytically (more complex).
• Validate with null (all effects 0) to estimate Type I error, and with specified effects for power.

This skeleton gives a reproducible simulator to estimate empirical Type I error and power under sequential monitoring with correlated metrics and an OF alpha-spending rule.