Junior Data Scientist Interview Preparation Guide - FAANG Standards

A correlated subquery references a column from the outer query inside its own WHERE (or other) clause, so conceptually the database re-runs it once per outer row, substituting that row's values each time. A non-correlated subquery has no reference back to the outer query at all, so it's evaluated once, independent of how many outer rows exist, and its result is reused for every row. Using employees(emp_id, department_id, salary), a correlated subquery can put each employee's department average right next to their own salary; a non-correlated subquery is the right tool for a single, outer-row-independent value like the company-wide average.

Correlated example: salary next to department average

SELECT
  e.emp_id,
  e.department_id,
  e.salary,
  (
    SELECT AVG(d.salary)
    FROM employees d
    WHERE d.department_id = e.department_id   -- correlated: references the outer row
  ) AS dept_avg_salary
FROM employees e
ORDER BY e.department_id, e.emp_id;

The inner query's WHERE d.department_id = e.department_id is what makes this correlated: e comes from the outer query, so the average has to be recomputed (conceptually) for every distinct department that appears in the outer rows.

Non-correlated example: a different, single-value comparison

SELECT emp_id, department_id, salary
FROM employees
WHERE salary > (
  SELECT AVG(salary) FROM employees   -- non-correlated: no reference to the outer row
);

The inner SELECT AVG(salary) FROM employees never mentions the outer table, so it produces one number, computed once, and every outer row is compared against that same value.

Key points

• The defining test is a reference, not the shape: a subquery becomes correlated the moment it uses a column from an outer table in its own filter or expression.
• A correlated scalar subquery in the SELECT list (as above) must return at most one row per outer row, or the database raises an error at runtime; the department-average query is safe because AVG() always collapses to one value.
• Real engines don't literally re-execute a correlated subquery row by row in every plan; a good optimizer can often rewrite it into a join or semi-join internally. "Conceptually re-runs per row" describes the logical semantics you should reason about when writing the query, not a guarantee about the physical execution plan.

Complexity

Logically, a correlated subquery costs O(rows in outer query) times O(cost of one inner-query execution). Whether that's actually slow depends entirely on whether the optimizer can turn it into a join or hash-based plan instead of a literal nested loop; an unindexed correlated lookup against a large table is the case where the naive per-row cost is real and painful. The non-correlated subquery costs O(1) execution of the inner query plus O(rows) for the outer comparison, since the inner result is just a constant once computed.

Worked example

With five employees across two departments (department 10: salaries 60000, 80000, 70000; department 20: salaries 50000, 90000), the correlated query above gives every department-10 employee a dept_avg_salary of 70000 and every department-20 employee 70000 as well (their department average happens to also be 70000: (50000+90000)/2). The non-correlated query, filtering against the company-wide average of 70000 ((60000+80000+70000+50000+90000)/5), returns exactly the two employees earning above it: the 80000 earner in department 10 and the 90000 earner in department 20. (Verified by executing both queries against SQLite 3.51 with this exact data.)

Trade-offs and pitfalls

For simple, single-value lookups and department-style row-level comparisons, either form works and correlated subqueries are often the most readable option. At production scale, prefer rewriting the correlated form as a JOIN against a pre-aggregated GROUP BY subquery or CTE (common table expression), or as an equivalent window function (AVG(salary) OVER (PARTITION BY department_id)), especially once you've checked EXPLAIN and confirmed the optimizer isn't already rewriting it for you; large, unindexed correlated subqueries are the classic case where a query silently degrades from fast in a small test dataset to slow in production.

Direct answer

Ownership should follow causal control, not who asked for the metric. A number that only one team's actions actually move belongs to that team as a leading indicator. A number that several teams jointly move needs to be treated as a shared outcome with exactly one canonical definition that everyone points to, not each team computing its own version of 'the same' number.

Structured elaboration

1. Decide ownership by who controls the number

Ask: if this metric moved tomorrow, whose decisions would most plausibly explain it? If the answer is one team, it's team-owned. If the honest answer is 'several teams, depending on the week,' it's a shared outcome metric and needs shared governance, not a single team's dashboard.

2. Give every shared metric one canonical definition

Store the computation (the query or transformation logic) in one place, documented with an owner, a last-updated date, and the exact filters and date logic used. Any dashboard or report showing that metric should read from that canonical source, not recompute it independently.

3. When two teams report different numbers, reconcile, don't debate

The canonical definition is the tiebreaker by default. If a mismatch appears, the fix is a reconciliation step: compare the two calculations side by side, find where the logic diverges (a different date window, a different filter, a stale cache), and correct the deviating one, or update the canonical definition itself if it turns out to be wrong. Either way, log the decision so the same disagreement doesn't restart from zero next quarter.

4. Put governance around who can change a shared definition

A shared metric's definition should not change because one team unilaterally decides a different cohort or window looks better. Route changes through a lightweight review involving everyone who reports on that metric, and version the definition so historical numbers can be explained if they shift after a redefinition.

Worked example

A dashboard spans data engineering, product, and marketing for a signup-to-paid-conversion initiative. Splitting ownership this way keeps the dashboard honest:

When marketing's report shows a different sign-up count than the shared dashboard, the reconciliation step finds that marketing's number excluded a promo-code cohort by mistake. The canonical query is correct; marketing's ad hoc report is fixed to match it, and the discrepancy is logged so the next person who notices a mismatch can find the resolution instead of reopening the debate.

Trade-offs and pitfalls

• Centralizing every metric, including team-level leading indicators, slows down the teams that need to iterate quickly on their own signals; only the genuinely shared outcomes need the heavier canonical-definition process.
• Fully decentralizing shared outcome metrics guarantees mismatched dashboards eventually, which quietly erodes trust in the data even when the underlying numbers are directionally fine.
• A 'single source of truth' only works if using an alternate calculation is treated as a defect to fix, not a valid difference of opinion; without that enforcement, teams drift back to their own numbers within a quarter.
• Late-arriving corrections that change historical values need an explicit policy (do dashboards restate history, or only apply corrections going forward) decided in advance, or every correction becomes its own dispute.

Pytest unit tests for normalize_email(email: str)

import pytest
from yourmodule import normalize_email

def test_strip_whitespace_and_lower_domain():
    assert normalize_email('  User+tag@EXAMPLE.COM  ') == 'User+tag@example.com'

def test_empty_string():
    assert normalize_email('') == ''

def test_trim_local_only():
    assert normalize_email('  local  @DoMain.Com') == 'local@domain.com'

def test_invalid_type_raises():
    with pytest.raises(TypeError):
        normalize_email(None)

Notes: tests cover leading/trailing whitespace, uppercase domain, empty string, and invalid type. Adjust expected behavior if function normalizes tags or lowercases local part.

Direct answer

A mutable type like Python's list can be changed in place through ANY reference that points to it, so two variables aliasing the same list see each other's edits. An immutable type like str can never be changed after creation, so every apparent "modification" (concatenation, slicing) actually allocates a brand new object, trading away in-place efficiency for the guarantee that nobody holding a reference to the original object can ever see it change under them.

Structured elaboration

• Aliasing and side effects. If alias = original for a mutable list, both names point at the SAME object; calling alias.append(x) changes what original sees too, because there is only one list, with two names. For an immutable str, s2 = s + "x" creates an entirely new string object; s itself is untouched, and s2 is a different object from the start.
• Memory. A single in-place list edit (like append) reuses the existing buffer (amortized O(1), ignoring occasional dynamic-array resizing). Every immutable-string "edit" allocates a new object, so a chain of n sequential edits on an immutable type creates and discards n intermediate objects along the way, even though only the final one survives, which is real (if transient) memory churn.
• Time complexity. Building a string one piece at a time with s += "x" in a loop copies the growing content into a new buffer on every iteration; building a string of length n this way costs 0 + 1 + 2 + ... + (n-1) = n(n-1)/2 character-copies in the general case, i.e. O(n^2) total, versus O(n) for a mutable buffer built once and joined (Python's "".join(...), or Java's StringBuilder).
• Thread-safety. An immutable object can be freely shared across threads with zero synchronization, because no thread can ever observe a change in it (there are none to observe). A mutable object shared across threads needs explicit synchronization (a lock, or a design that simply avoids sharing it), or concurrent mutation causes a data race.
• Production ML pipelines, specifically. Feature-engineering pipelines commonly pass a large shared list, array, or dataframe reference between transform steps to avoid copying big data on every stage, which is fine as long as (a) each stage either only mutates its OWN data or clearly documents that it mutates the shared object in place, and (b) nothing downstream still expects to see the PRE-mutation version. A common real bug: a stage keeps what it believes is a "before" snapshot for a data-quality check or an A/B comparison, but that "before" is actually the SAME list object a later stage then mutates in place, so the "before" silently becomes the "after" too. A common production pattern to avoid this: mutate large mutable buffers in place freely WITHIN a single stage for performance, but treat data crossing a stage or worker boundary as effectively immutable (copy on boundary, or hand off an immutable type like a tuple or a frozen structure), so no stage can accidentally corrupt another stage's view of the data. This also matters for multiprocessing: passing immutable data between worker processes is inherently race-free (each process gets its own copy on the way in), while mutable shared state across process boundaries needs explicit shared-memory handling to avoid corruption.
• When copying is safer, in general. Whenever the SAME object needs to outlive the point of mutation and something else still needs the original: checkpoints, before/after diffs, a retry that must restart from the original input, or parallel workers that must not race on the same buffer.
• Java and C++. Java's String is immutable in exactly the same way as Python's str (repeated + concatenation pays the same new-object-per-step cost; StringBuilder is the same fix as join/building a list of parts). Java's ArrayList and C++'s std::vector are mutable like Python's list, with the same aliasing behavior (two references to the same vector see each other's changes) and the same discipline of copying deliberately when a callee must not affect the caller's data. C++ additionally encodes this choice directly in the type signature (pass-by-value copies, pass-by-reference or pointer aliases), rather than Python's single reference-semantics-for-everything model, which makes the choice more visible at the call site but not fundamentally different in kind.

Worked example

def append_item(container, item):
    container.append(item)  # mutates in place, no reassignment

original = [1, 2, 3]
alias = original
append_item(alias, 4)
print(f"original list after mutating alias: {original}")
assert original == [1, 2, 3, 4]
assert alias is original
print(f"alias is original (same object): {alias is original}")

def chars_copied_plus_equals(n):
    # deterministic cost model, not a wall-clock timing claim: if every '+='
    # allocates a new buffer and copies the existing content into it (the
    # general immutable-string contract), building length n one char at a
    # time copies 0+1+...+(n-1) = n(n-1)/2 characters total: O(n^2)
    total, length_so_far = 0, 0
    for _ in range(n):
        total += length_so_far
        length_so_far += 1
    return total

for n in (10, 100, 1000):
    plus_cost = chars_copied_plus_equals(n)
    formula = n * (n - 1) // 2
    assert plus_cost == formula
    print(f"n={n}: naive '+=' copies {plus_cost} chars total (n(n-1)/2={formula}), join copies {n} chars total")

Output (executed, python3 s71_mutable_immutable.py):

original list after mutating alias: [1, 2, 3, 4]
alias is original (same object): True
original string unchanged: 'abc', new string returned: 'abcd'
n=10: naive '+=' copies 45 chars total (n(n-1)/2 = 45), join copies 10 chars total
n=100: naive '+=' copies 4950 chars total (n(n-1)/2 = 4950), join copies 100 chars total
n=1000: naive '+=' copies 499500 chars total (n(n-1)/2 = 499500), join copies 1000 chars total

The copy-count model, not a wall-clock benchmark, is what demonstrates the O(n^2) versus O(n) gap here: at n=1000 the naive approach has already copied roughly 500x as many characters as join would, and that ratio keeps growing with n.

Trade-offs & pitfalls

• Assuming a copy happened because a method LOOKS like it should return something new is a real trap: list.sort() mutates in place and returns None, while a full slice arr[:] DOES return a copy; the two are easy to confuse.
• Silent aliasing bugs are usually invisible until two pieces of code happen to run in a particular order that exposes them, a notorious source of "it worked when I ran my notebook cells in this order, but not when I re-ran the whole thing top to bottom" bugs in exploratory ML code.
• Over-defensive copying everywhere is its own real cost (wasted memory and time on large arrays); the right discipline is deliberate copying at genuine ownership boundaries, not reflexive copying of everything just in case.
• Note on the timing claim above: CPython specifically has an internal optimization that can special-case a single-reference string being repeatedly concatenated, making += faster in practice than the worst-case cost model on THAT one interpreter. It is not part of the language specification, does not apply once the string has more than one reference, and is not present in other Python implementations or other languages, so the join/StringBuilder-style pattern remains the portable, guaranteed-safe habit rather than something to skip on the assumption that CPython always handles it for you.

Direct answer: In one case, two pipelines used the same field name ("user_id" or an equivalent) to mean subtly different things, and I noticed it because a downstream metric that should have matched across two reports didn't, which led me to trace the discrepancy back to a semantic mismatch rather than a computation bug.

Structured elaboration (STAR):
Situation: two teams each maintained their own version of a user-level metric with the same name, and a cross-team report combining both started showing numbers that didn't reconcile. Task: figure out why two supposedly-identical metrics disagreed, and fix it without breaking either team's existing consumers. Action: I compared the actual transformation logic behind each metric (not just the documentation, which agreed) and found one team's version excluded a specific user segment the other included, a distinction that had never been made explicit anywhere the two teams could see it. Result: I proposed renaming both to make the distinction explicit in the name itself, documented the difference in the shared feature catalog, and set up an automated check comparing the two going forward so a future silent divergence would be caught immediately rather than discovered by accident in a downstream report.

Worked example: The concrete tell that something was wrong wasn't a crash or an error, it was two numbers that SHOULD have matched (by every piece of documentation) quietly not matching in a routine cross-team reconciliation report, which is exactly the kind of subtle, non-crashing signal that this class of bug tends to produce.

Trade-offs and pitfalls: The fix that actually mattered wasn't just correcting the immediate discrepancy, it was making the underlying ambiguity impossible to silently recur, since renaming and documenting without an ongoing automated check would have left the same failure mode free to happen again the next time either team's definition drifted.

Direct answer

Reverse the whole array, then reverse the first k elements and the remaining n-k elements separately; three linear passes compose into the fully rotated result with no auxiliary array. The same reversal trick generalizes directly: reversing a string in place is the identical two-pointer, swap-from-both-ends routine, and rotating a square matrix 90 degrees in place is a transpose followed by reversing each row, both built on the same in-place-swap primitive as the array rotation.

Structured elaboration

Why three reversals produce a rotation. Reversing the entire array puts every element in fully reversed order. Reversing the first k elements of that reversed array un-reverses exactly the block that should now sit at the front, restoring its original relative order; reversing the remaining n-k elements does the same for the remainder. Normalizing with k %= n handles k values larger than the array's length or equal to zero.

Generalizing to a string. The same in-place two-pointer swap from both ends is exactly what reverses a string, provided the string is held in a mutable container (a list of characters, for example, since Python's own string type is immutable and cannot be reversed truly in place without first converting it).

Generalizing to a square matrix. Transposing swaps matrix[i][j] with matrix[j][i] for every i < j, turning rows into columns. Reversing each row afterward flips left to right. Combined, what was the first column read top to bottom becomes the first row read left to right, which is exactly a 90-degree clockwise turn.

Related in-place-preprocessing techniques (with an honest space caveat). Prefix-sum preprocessing builds an auxiliary array once, in $O(n)$ time, so that any later range-sum query answers in $O(1)$; this trades $O(n)$ extra space for fast queries, so it is not itself an O(1)-extra-space technique, even though it shares this family's "one linear pass, reuse the result" character. Product-except-self, by contrast, genuinely can be done with O(1) extra space beyond the required output array: a first pass fills the output with the running product of everything to each index's left, and a second pass multiplies in the running product of everything to that index's right, needing no separate auxiliary array at all.

Worked example

def rotate_array(nums: list[int], k: int) -> None:
    n = len(nums)
    if n <= 1:
        return
    k %= n
    if k == 0:
        return

    def reverse(i, j):
        while i < j:
            nums[i], nums[j] = nums[j], nums[i]
            i += 1
            j -= 1

    reverse(0, n - 1)
    reverse(0, k - 1)
    reverse(k, n - 1)


def reverse_string_inplace(chars: list[str]) -> None:
    i, j = 0, len(chars) - 1
    while i < j:
        chars[i], chars[j] = chars[j], chars[i]
        i += 1
        j -= 1


def rotate_matrix_90_cw_inplace(matrix: list[list[int]]) -> None:
    n = len(matrix)
    for i in range(n):
        for j in range(i + 1, n):
            matrix[i][j], matrix[j][i] = matrix[j][i], matrix[i][j]
    for row in matrix:
        row.reverse()


if __name__ == "__main__":
    arr = [1, 2, 3, 4, 5, 6, 7]
    rotate_array(arr, 3)
    print(arr)

    chars = list("hello")
    reverse_string_inplace(chars)
    print("".join(chars))

    m = [[1, 2, 3], [4, 5, 6], [7, 8, 9]]
    rotate_matrix_90_cw_inplace(m)
    print(m)

Running this prints [5, 6, 7, 1, 2, 3, 4], then olleh, then [[7, 4, 1], [8, 5, 2], [9, 6, 3]].

Complexity

rotate_array: time $O(n)$ for the three reversal passes, since they compose additively into a single linear scan rather than multiplying; space $O(1)$ extra, using only the two index pointers inside each reversal call.
reverse_string_inplace: time $O(n)$, one pass with two pointers closing in from both ends; space $O(1)$ extra beyond the mutable character list itself.
rotate_matrix_90_cw_inplace: time $O(n^2)$ for an n-by-n matrix, since the transpose visits each of the $n^2$ cells once; space $O(1)$ extra, since both the transpose and the row reversals swap in place with no auxiliary matrix.

Edge cases

• k = 0, or k a multiple of the array's length once normalized via k %= n: rotate_array detects this and returns immediately without performing any reversals, since the array is already in its correct rotated position.
• Empty or single-element array or string: both rotate_array (via its n <= 1 guard) and reverse_string_inplace (via while i < j never firing) return immediately with nothing to do.
• A non-square matrix passed to rotate_matrix_90_cw_inplace: this implementation assumes a square matrix, and a non-square transpose changes the matrix's dimensions, so it cannot be rotated true in place this way.

Trade-offs & pitfalls

Forgetting k %= n for a k larger than the array's length either wastes work or, in a careless implementation, indexes out of range. The transpose-then-reverse-rows trick only works for a square matrix: transposing a non-square matrix changes its dimensions, so a genuinely non-square rotation needs a separate output buffer rather than a true in-place transform. Python's string immutability means a real in-place string reversal needs a mutable container (a list of characters, or a bytearray) first; there is no way to mutate a str object's characters directly.

Situation: Our team of three currently uses scikit-learn; leadership wants us to adopt tf.keras for deep-learning needs and model deployment consistency.

Task: Lead the upskilling and validate adoption via an 8-week pilot so the team can reliably build, evaluate, and deploy tf.keras models.

Action:

• Assess skills (week 0): run a short intake:
  • Self-assessment survey (experience with NN concepts, TensorFlow, GPU, Docker).
  • 2-hour coding assessment: build/train a simple classifier (deliverable: repo + README).
  • Review recent code to identify gaps (data pipelines, TF idioms).
• Design pilot project:
  • Choose a focused, production-relevant task (e.g., migrate an existing tabular churn model to a small neural net in tf.keras with same dataset, add a simple NN for embeddings or feature crosses).
  • Success criteria: model parity or improvement on business metric, reproducible training pipeline, and containerized deployment.
• Curate learning resources & run workshops:
  • Week 1–2: fundamentals workshop (tf.keras API, layers, callbacks, training loops) — 2x 90-min code-along sessions with notebooks.
  • Week 3: data pipeline & tf.data hands-on.
  • Week 4: model debugging, metrics, and unit testing for ML.
  • Week 5: scaling, GPU usage, and saving/loading models (SavedModel, TF Serving).
  • Week 6: CI/CD + container deployment demo.
  • Provide curated materials: official TensorFlow guides, short videos, cheat-sheets, example repos, and a “starter kit” repo with templates.
  • Pair programming: rotate pairing so each member leads a mini-task.
• Milestones for 8-week pilot:
  • Week 1: assessment complete; baseline model and starter repo initialized.
  • Week 2: team completes fundamentals workshops; notebook exercises submitted.
  • Week 3: prototype tf.data pipeline + preprocessing saved as reusable module.
  • Week 4: first tf.keras model trained end-to-end; basic evaluation metrics produced.
  • Week 5: hyperparameter tuning experiment + reproducible training script.
  • Week 6: containerized inference endpoint + simple load test.
  • Week 7: compare performance vs scikit-learn baseline; document migration steps.
  • Week 8: final demo to stakeholders, retrospective, decision on adoption next steps.
• Metrics to measure adoption:
  • Learning/proficiency: pre/post assessment scores and completion rate of code exercises (target: +30% improvement).
  • Productivity: time to reproduce training run (target: ≤ baseline or improved by x%), number of PRs using tf.keras templates.
  • Model quality: parity or improvement on business metric (e.g., AUC or lift within ±2% or better).
  • Engineering readiness: presence of automated training script, SavedModel artifacts, container image, and end-to-end test coverage (target: 80% of pilot repo components have CI tests).
  • Operational metrics: inference latency and CPU/GPU utilization within budget.
  • Behavioral: number of internal demos taught, documentation pages authored, and team confidence in a follow-up survey (target: majority comfortable to use tf.keras independently).
• Risks & mitigations:
  • If stuck on week 3 pipeline issues — schedule focused debug session with a TF expert or use simplified dataset.
  • If model lags substantially — document trade-offs and propose hybrid approach (scikit-learn for simple models, tf.keras for deep models).

Result: At pilot end we’ll have objective data (assessments, repo, deployed endpoint, and stakeholder demo) to decide organization-wide adoption and a clear onboarding path for others. This approach balances learning, hands-on experience, measurable outcomes, and minimal disruption to ongoing work.

Direct answer. Vectorize the comparisons with boolean masks (or a single pass of counters) to get true O(n), and guard every division with a zero check so an empty positive-prediction set or an empty positive-label set returns 0.0 instead of raising.

Structured elaboration. The four confusion-matrix quantities are just boolean AND-reductions over paired arrays; accuracy needs the total, precision needs TP+FP, recall needs TP+FN, and F1 is the harmonic mean of precision and recall. The only real edge-case work is deciding what "0/0" should mean for each of precision, recall, and F1 (convention below: 0.0, not NaN, so a caller doing arithmetic on the result never gets contaminated by a NaN).

Code (executed and verified below).

import numpy as np

def compute_classification_metrics(y_true, y_pred):
    y_true = np.asarray(y_true)
    y_pred = np.asarray(y_pred)
    tp = int(np.sum((y_true == 1) & (y_pred == 1)))
    fp = int(np.sum((y_true == 0) & (y_pred == 1)))
    fn = int(np.sum((y_true == 1) & (y_pred == 0)))
    tn = int(np.sum((y_true == 0) & (y_pred == 0)))
    accuracy = (tp + tn) / len(y_true) if len(y_true) else 0.0
    precision = tp / (tp + fp) if (tp + fp) else 0.0
    recall = tp / (tp + fn) if (tp + fn) else 0.0
    f1 = 2 * precision * recall / (precision + recall) if (precision + recall) else 0.0
    return {"TP": tp, "FP": fp, "FN": fn, "TN": tn,
            "accuracy": accuracy, "precision": precision, "recall": recall, "f1": f1}

Worked example (recomputed, not eyeballed). y_true=[1,0,1,1,0,1,0,0,1,0], y_pred=[1,0,0,1,0,1,1,0,1,0] gives TP=4, FP=1, FN=1, TN=4, accuracy = 8/10 = 0.8, precision = 4/5 = 0.8, recall = 4/5 = 0.8, F1 = 0.8 (all four numbers agree because precision and recall happen to be equal here). Edge cases: y_true=[1,0,1,0], y_pred=[0,0,0,0] (model never predicts positive) returns TP=0, FP=0 so precision falls back to 0.0 rather than dividing 0/0; y_true=[0,0,0,0], y_pred=[1,0,0,0] (no positive labels at all) returns FN=0 so recall also falls back to 0.0 cleanly.

Trade-offs and pitfalls. The four boolean-mask sums are each a single vectorized pass, so the whole function is O(n) time and O(1) extra space beyond the output; a naive Python loop with per-element branching would also be O(n) but roughly 20 to 50 times slower in practice due to per-element interpreter overhead. Returning 0.0 for an undefined precision or recall is a convention, not a law. Some teams prefer to return NaN to make "undefined" visibly distinct from "genuinely zero"; whichever you choose, document it, because silently treating "no positive predictions" the same as "predicted everything correctly as negative" can hide a broken model from an automated monitor that only checks whether precision dropped below a floor.

Direct answer

Document the finding precisely (which attribute, how strong the correlation, and with what protected characteristic or plausible proxy), run basic fairness checks (does the outcome actually differ by the protected group, not just the proxy) before anyone builds on this data, and loop in whoever owns compliance, legal, or fairness review for the domain, especially in a regulated context like lending or hiring. Don't quietly drop the feature and move on without documenting why, and don't quietly keep using it either.

Why this needs more than a technical fix

A correlation between a proxy variable and a protected attribute doesn't automatically mean the feature is unusable, but it does mean anyone building on this data needs to know the risk exists, and the decision about how to handle it (remove the feature, transform it, apply a fairness constraint, or accept it with documented justification) is a judgment call that usually shouldn't be made unilaterally by whoever happens to notice it during EDA.

Worked example

Exploring features for a lending scoring model, ZIP code turns out to correlate strongly with race in this dataset, a well-known real-world pattern given historical housing segregation. Documenting the finding: which ZIP codes are involved, the strength of the correlation, and a note that this is a widely-recognized proxy risk in lending specifically. The next step is checking whether the MODEL'S predicted outcome differs meaningfully by protected group once ZIP is included, not just whether ZIP itself correlates with race, since the practical risk is in the downstream decision, not the raw correlation alone. This finding then goes to the team's fairness or compliance reviewer, along with the specific evidence, rather than being resolved solely by removing the feature and hoping the risk goes away with it.

Trade-offs and pitfalls

Removing an obviously-proxying feature doesn't guarantee fairness on its own, since other features can jointly encode similar information even after the most obvious proxy is gone; that deeper mitigation work (choosing and optimizing for a specific fairness metric, applying a fairness constraint during training) is a distinct, more involved effort than the EDA-stage discovery and escalation described here.

Direct answer

Treat the blocked dependency as three separate jobs, not one: propose a specific, swappable alternative (never "we'll find another data source"), quantify how much that alternative actually costs you against the same basis as your original goal, and only then communicate it to stakeholders as a decision they need to accept, with a plan to re-check the estimate once the workaround is live rather than trusting it forever.

Structured elaboration

1. Confirm and scope the blocker precisely. Is the resource permanently gone, temporarily rate-limited, or genuinely nonexistent? That changes whether you're building a permanent workaround or a temporary bridge.

2. Propose concrete, named alternatives. Generate two or three real options: a specific proxy resource or feature, a reduced-scope version of the original approach, or a phased plan that defers the dependency-blocked piece. Vague alternatives ("we'll look into it") aren't actually decisions stakeholders can evaluate.

3. Quantify trade-offs and confidence on the same basis as the original success metric. State an actual number and an uncertainty band for each option, derived the same way you'd have measured the original approach, not a rough feeling.

4. Communicate and validate. Present the trade-off with numbers, get explicit sign-off on the accepted floor, and set a checkpoint to re-measure the workaround against real data once it's live, with a defined rollback trigger if it underperforms.

Worked example: I was building a churn-prediction model and had planned to use "days since last support ticket" as a key input, sourced from the support-ticketing system's application programming interface (API). Ten days into a four-week build, I discovered the API only retains 30 days of ticket history and permanently deletes older records, a hard constraint, not a temporary outage. Alternative proposed: "count of support tickets in the last 30 days" (available from the same API) combined with a coarser "has any ticket history" flag pulled from a separate monthly business-intelligence export that holds 18 months of history. Quantified trade-off: on a 5,000-customer holdout where the full history happened to still exist in an old export, the original feature had a 0.34 point-biserial correlation (a -1 to 1 score of how strongly this one feature tracks with churn, read the same way as an ordinary correlation coefficient) with churn (measured over the same 90-day churn label window as everything else here); the proxy set had 0.29, about a 15% relative drop in that single feature's signal. In cross-validation, projected model accuracy, measured as area under the ROC curve (AUC, a 0 to 1 score of how well the model separates customers who churn from those who don't), dropped from 0.81 with the original feature to a measured 0.78 with the proxy set, a 0.03 point drop. I flagged plainly that the 0.81 number was a projection from an older pilot, not measured on the same holdout, so it wasn't a fully apples-to-apples comparison, only the 0.78 was a real measurement. Communicate and validate: I presented that trade-off table to the model's business owner and the data governance lead within two days of finding the blocker, got sign-off on a 0.78 floor (against a previously agreed minimum bar of 0.75), and set a validation checkpoint at 60 days post-launch to re-measure actual AUC on live traffic, with a rollback trigger if it fell below 0.73.

Second example (different discipline): a solutions architect planning a batch workload discovers mid-build that the target cloud region has no capacity for the planned GPU (graphics processing unit) instance type for six weeks. The alternative: split the workload across two smaller GPU instances in a different region. Quantified trade-off: a 22% increase in projected compute cost (same on-demand hourly rate basis, same workload size) and a 40-minute increase in total job runtime, measured by running the actual job once on the same test dataset before committing every future run to it, then communicating those specific numbers to the finance and program stakeholders instead of "it'll cost a bit more."

Trade-offs and pitfalls

The mediocre move is "I'd look for another data source and let people know," which names no concrete proxy, no quantified confidence, and no validation step, so it can't actually be evaluated or shown to be wrong later. The subtler trap is presenting a pre-launch estimate as if it settles the question forever; quantifying confidence once and never checking it against reality is how a workaround quietly becomes a permanent, unverified assumption.