HardTechnical

34 practiced

During feature or dataset preparation for a supervised model, a join can accidentally introduce label leakage: for example, joining in a table that contains information only available after the outcome is known, or a join that looks ahead in time. Describe common sources of this kind of leakage, how you would detect it automatically, and how you would design a rollback or re-computation process once leakage is found.

Sample Answer

Direct answer Join-induced label leakage happens when a feature you join into a training dataset was only available AFTER the outcome you're predicting was already known, so the model "cheats" by learning a feature that is really just a proxy for the label. The fix is systematic: for every joined feature, check whether its timestamp could postdate the label event, not just whether the join key is correct. Structured elaboration Common sources: joining in a "resolution" or "outcome" table (chargeback filed, ticket closed, refund issued) that is only populated after the event you're predicting has already happened; joining a snapshot of a dimension table AS OF today rather than as of the training example's own event time, which lets future state leak backward; and joins that bring in aggregates computed over a window that includes the label event itself. Detection: for every feature, ask "could this column's value have existed at prediction time, for this specific row?" A concrete automatable check is comparing the feature's own timestamp (or the timestamp of the table it came from, at ingestion) against the label event's timestamp, and flagging any feature that is not strictly BEFORE it. A telling symptom: a feature that correlates almost perfectly with the label is a strong practical signal to inspect first, real predictive features are rarely that clean. Remediation once found: remove the offending join or rebuild it to only pull the AS-OF-prediction-time value (a point-in-time correct join, not a "latest" join); re-train and re-validate that accuracy drops to something more realistic, which paradoxically is the correct outcome of the fix. Process control: automate the timestamp check as a standing test in the training pipeline (not a one-time manual audit), since leakage tends to reappear whenever a new feature or a new upstream table is added by someone unaware of this specific pipeline's history. Worked example pythonimport pandas as pd labels = pd.DataFrame({'txn_id':[1,2,3], 'is_fraud':[0,1,0]}) resolutions = pd.DataFrame({ 'txn_id':[1,2,3], 'chargeback_filed':[False, True, False], # only known AFTER investigation concludes 'resolved_at':['2024-02-01', None, '2024-02-01'], }) merged = labels.merge(resolutions, on='txn_id') Here chargeback_filed is perfectly correlated with is_fraud (verified: every row where is_fraud=1 has chargeback_filed=True and vice versa) precisely because it is set by the same investigation process that determined the label; a model trained with this feature would show excellent offline accuracy and then fail in production, where chargeback_filed doesn't exist yet at prediction time. Trade-offs and pitfalls A feature does not need to be a literal copy of the label to leak; a near-perfect correlation is the warning sign, but a subtler leak (an aggregate that includes the label event in its own window) can hide behind a merely "very good" correlation instead of a perfect one. Removing a leaky feature and watching accuracy drop is not a failure, it is the fix working; be prepared to explain that to a stakeholder who saw the earlier (wrong) number and now sees a lower one. Automated point-in-time joins are more expensive to build and maintain than a simple "latest value" join; the cost is worth it specifically for any feature that could plausibly reflect a post-label-event fact.

HardTechnical

37 practiced

A transformation job produced different output across two runs even though the code did not change. Walk through the debugging steps you would take to find the cause, and what you would put in place afterward to prevent a recurrence.

Sample Answer

Direct answer Non-deterministic transformation output almost always traces to an operation whose result depends on an ordering the system doesn't guarantee, most commonly an unordered groupby().first()/.last() on data whose row order varies between runs; the fix is to make the intended ordering explicit (sort by a real, meaningful key before the operation) rather than relying on whatever order the data happened to arrive in. Structured elaboration Confirm it's really non-determinism, not a data change: first verify the exact same input snapshot produces different output across runs; if the input itself silently changed between runs, that's a different bug (a data-freshness issue), not a non-determinism issue. Check dependency and environment versions: a library upgrade can silently change tie-breaking behavior in an operation that was never guaranteed to be stable in the first place. Look specifically for unordered operations: groupby().first(), .drop_duplicates() without a prior sort, and any join or shuffle that doesn't have a secondary deterministic tie-break are the recurring culprits; each of these has a "first/any" semantic that depends on row order unless you make that order explicit yourself. Check for unseeded randomness: any sampling, shuffling, or randomized algorithm needs an explicit, fixed seed for its output to be reproducible; forgetting this is a second common source of run-to-run variation. The fix, once found: sort by an explicit, meaningful key (a timestamp, a stable id) BEFORE the aggregation that assumed an order; this makes "first" and "last" actually mean something specific and repeatable, rather than "whatever happened to be first in an arbitrary in-memory order." Worked example pythonimport pandas as pd df = pd.DataFrame({'user_id':[1,1,2,2], 'event':['A','B','C','D'], 'val':[10,20,30,40]}) shuffled1 = df.sample(frac=1, random_state=1).reset_index(drop=True) shuffled2 = df.sample(frac=1, random_state=2).reset_index(drop=True) first1 = shuffled1.groupby('user_id')['event'].first() first2 = shuffled2.groupby('user_id')['event'].first() Verified: with the SAME logical data shuffled two different ways (simulating two runs where input row order isn't guaranteed), groupby('user_id')['event'].first() returned DIFFERENT results across the two shuffles ({1: 'A', 2: 'D'} versus {1: 'B', 2: 'C'}), concretely reproducing the bug class. Adding an explicit .sort_values('ts') by a real timestamp column before the groupby, and re-running against the same two differently-shuffled inputs, produced IDENTICAL results both times, confirming the fix actually addresses the root cause rather than merely appearing to. Trade-offs and pitfalls This class of bug is dangerous specifically because it's silent: the code runs successfully every time, produces plausible-looking output every time, and only the SPECIFIC VALUE differs between runs, which is much harder to catch in code review or a single test run than an outright crash. A test that only runs an aggregation once against a fixed, already-sorted test fixture will never catch this bug, since the fixture never exercises the ordering-dependence at all; a good regression test for this class of issue should specifically shuffle the input and assert the output is stable across multiple different shuffles. The general lesson generalizes beyond groupby().first(): any time a transformation's correctness silently depends on an ordering guarantee the underlying engine doesn't actually promise, the same class of bug is possible, sort explicitly wherever the result should be deterministic.

HardTechnical

33 practiced

Some aggregations (sum, count) are associative and trivially parallelizable across a distributed dataset with a guaranteed deterministic result. Others, like median or percentile, are not. Discuss how you would compute an approximate percentile at scale with a mergeable, deterministic algorithm (for example a t-digest or histogram sketch), and separately, how you would implement a scalable approximate 'distinct count' (for example unique users in the last 30 days) using a structure like HyperLogLog, including the accuracy/memory trade-off of each.

Sample Answer

MediumTechnical

34 practiced

Implement a discount or promotion rule as deterministic, testable logic: for example, a line item qualifies for a discount only when several conditions all hold (a flag on the customer, a threshold on the order total, and a lookback window with no qualifying event). When multiple promotions could apply to the same item, the best discount should win. Provide an implementation (SQL or pseudocode) and describe how you would unit-test the rule set, including its edge cases.

Sample Answer

MediumTechnical

40 practiced

You receive phone numbers from several partners in inconsistent formats (with or without country codes, extensions, punctuation, or leading zeros). Design a normalization and validation workflow that produces a single canonical format suitable for matching and analytics, including how you would flag numbers you cannot confidently normalize and any privacy considerations for storing them.

Sample Answer

Direct answer Normalize confidently-parseable phone numbers into a single canonical format (E.164-style, a leading + and country code), extract any extension separately rather than losing it or corrupting the main number, and explicitly return "cannot confidently normalize" for genuinely ambiguous inputs instead of guessing a country code. Structured elaboration Strip formatting, isolate digits: remove punctuation, spaces, and parentheses to get the raw digit sequence, then separately detect and extract an extension (x123, ext. 123) if present, before trying to interpret the remaining digits as a phone number. Confident cases: a number that already starts with + is assumed to include an explicit country code; a 10-digit number (in a context where you know the default country, e.g. US) can be confidently prefixed with that default country code; an 11-digit number starting with a 1 similarly maps cleanly to a US/Canada E.164 number. Ambiguous cases, fail closed: a number like 001-44-20-1234 mixes an exit code convention with a country code in a way that isn't safely disambiguated without knowing the sender's own dialing convention; rather than guess, return "cannot confidently normalize" and route it to a review queue. Privacy consideration: phone numbers are personal data; whatever normalization and storage approach you use should follow the same access-control and retention discipline as other PII, not be treated as a harmless free-text field just because it's short. Worked example pythondef canonicalize_phone(raw, default_country='1'): digits = re.sub(r'\D', '', raw) ext = None m = re.search(r'(?:x|ext\.?)\s*(\d+)$', raw, re.IGNORECASE) if m: ext = m.group(1) digits = re.sub(r'\D', '', raw[:m.start()]) if raw.strip().startswith('+'): return '+' + digits, ext if len(digits) == 10: return '+' + default_country + digits, ext if len(digits) == 11 and digits.startswith('1'): return '+' + digits, ext return None, ext # cannot confidently normalize Verified: '+1 (555) 123-4567' and a bare '5551234567' (with a known US default) both correctly normalize to +15551234567; '5551234 x123' correctly extracts extension 123 while returning None for the main number (a 7-digit number is too ambiguous to confidently assign a country code to); '001-44-20-1234' correctly returns None rather than a confidently-wrong guess. Trade-offs and pitfalls Assuming a default country for any bare 10-digit number is only safe when you actually know the data source's country context; applying a US default to a dataset that actually mixes multiple countries' numbers will silently mis-normalize a real fraction of them. A library purpose-built for phone number parsing (handling the full space of international dialing conventions correctly) is generally a better production choice than a hand-rolled regex approach like the one above, which is presented here as an illustration of the fail-closed PRINCIPLE, not a claim that this exact regex handles every real-world case. Storing a normalized phone number without also retaining (or being able to reconstruct) the original raw input can make it hard to audit or re-normalize later if the canonicalization rule itself needs to change.

Data Transformation and Processing Logic Questions

Sample Answer

Direct answer

Structured elaboration

Worked example

Trade-offs and pitfalls

Sample Answer

Direct answer

Structured elaboration

Worked example

Trade-offs and pitfalls

Sample Answer

Direct answer

Structured elaboration

Worked example

Trade-offs and pitfalls

Sample Answer

Direct answer

Structured elaboration

Worked example

Trade-offs and pitfalls

Sample Answer

Direct answer

Structured elaboration

Worked example

Trade-offs and pitfalls

Unlock Full Question Bank