Data Quality and Bias Questions

Covers both the conceptual and technical aspects of data quality assessment, bias identification, and remediation. Candidates should be able to recognize common sources of bias including selection bias, confirmation bias, measurement bias, and sample limitations, and describe how these biases and methodological limitations affect conclusions. They should be able to document and communicate caveats and limitations clearly and responsibly. On the technical side, candidates should demonstrate techniques for detecting and handling missing values, duplicates, outliers, and inconsistent data types; explain trade offs between filtering, imputing, and transforming data; and discuss how data cleaning choices influence downstream analysis. Additional expected skills include validating cleaned data against expectations, performing sensitivity analyses to show how results change under different data handling decisions, tracking data provenance, and describing reproducible processes for data quality management.

EasyTechnical

89 practiced

You run temperature sensors across warehouses and notice that some sensors slowly drift upward over months. Describe how you would detect measurement bias using historical data and external references, and propose remediation strategies: recalibration, model-based correction, automated exclusion rules, and monitoring.

Sample Answer

Approach (overview): I’d treat this as a time-series measurement-bias detection and correction problem using internal historical baselines and external references (trusted sensors or weather stations). Goals: detect gradual drift, quantify bias, correct or quarantine sensors, and maintain ongoing monitoring.

Detection- Baseline drift test: compute each sensor’s residual = sensor_t − fleet_median_t (or location-specific reference) over rolling windows (30/90 days). Fit a linear trend to residuals; significant positive slope (p < 0.01) indicates upward drift.- External reference comparison: align timestamps with nearest calibrated reference (master sensor or external station). Compute long-term bias = mean(sensor − reference) and trend.- Statistical controls: use control charts (CUSUM or EWMA) on residuals to detect small, persistent shifts faster than simple thresholds.- Quantify uncertainty: bootstrap confidence intervals for slope and mean bias to avoid overreacting to noise.

Remediation strategies1. Recalibration (physical):- Trigger when bias magnitude > acceptable threshold (e.g., >0.5°C or >2× sensor noise) or trend significant.- Prioritize high-impact locations; schedule maintenance and record pre/post calibration offsets for audit.

2. Model-based correction:- Learn a correction model per sensor or per cohort (e.g., linear drift model, spline, or Kalman filter) using historical residuals vs. reference.- Apply real-time bias correction: corrected = raw − estimated_bias(t).- Retrain periodically and include covariates (ambient humidity, battery voltage, sensor age) if correlated with drift.

3. Automated exclusion rules:- Define rules to mark data as suspect automatically: e.g., persistent upward trend for >30 days, bias > threshold vs. reference, sudden variance increase.- Tag data as “quarantined” for downstream consumers; allow manual override after inspection.

4. Monitoring and alerting:- Dashboards showing per-sensor residuals, trend slopes, control-chart signals, calibration history.- Automated alerts (email/SMS) when CUSUM/EWMA crosses thresholds or when many sensors in a region drift (possible systemic issue).- KPIs: % sensors within spec, mean fleet bias, detection-to-resolution time.

Evaluation and trade-offs- Validate corrections by holding out reference periods and measuring RMSE reduction and bias removal.- Beware overfitting corrections—prefer simple parametric drift models unless data justify complexity.- Balance cost: frequent recalibration is costly; model corrections plus targeted recalibration often optimal.

This pipeline provides fast detection, defensible correction, and operational controls to keep sensor data trustworthy.

MediumTechnical

88 practiced

Write a Python function that, given a pandas Series of numeric values, flags outliers using the Median Absolute Deviation (MAD) method. Function signature: def flag_outliers_mad(series: pd.Series, thresh: float = 3.5) -> pd.Series. Explain complexity and why MAD is robust to extreme values.

Sample Answer

Approach: compute the median, the median absolute deviation (MAD), convert deviations to a modified z-score (using constant 0.6745 so MAD estimates std for normal data), and flag points with |modified_z| > thresh. Use vectorized pandas/numpy operations for O(n) time.

python

import pandas as pd
import numpy as np

def flag_outliers_mad(series: pd.Series, thresh: float = 3.5) -> pd.Series:
    """
    Flag outliers in a numeric pandas Series using the MAD (Median Absolute Deviation) method.
    Returns a boolean Series aligned with the input index: True = outlier, False = normal.
    thresh: threshold on the modified z-score (common default: 3.5)
    """
    # Work on a numeric copy; preserve index
    x = series.astype(float)
    
    # Keep NaNs in place: result will be False for NaNs
    nan_mask = x.isna()
    
    # Median
    med = x.median()
    
    # Absolute deviations from median
    abs_dev = (x - med).abs()
    
    # MAD: median of absolute deviations
    mad = abs_dev.median()
    
    # Constant to make MAD comparable to standard deviation for normal data
    c = 0.6745
    
    # If mad == 0 (e.g., constant series), avoid division by zero:
    # In that case, consider there are no MAD-based outliers (could also fallback to IQR)
    if mad == 0 or np.isclose(mad, 0.0):
        outliers = pd.Series(False, index=series.index)
        outliers[nan_mask] = False
        return outliers
    
    # Modified z-score
    modified_z = (c * (x - med)) / mad
    
    # Flag outliers where absolute modified z-score exceeds threshold
    outliers = modified_z.abs() > thresh
    
    # Ensure NaNs are not flagged
    outliers[nan_mask] = False
    
    return outliers

Key points:- Time complexity: O(n) — single-pass vectorized operations (median and elementwise arithmetic).- Space complexity: O(n) for intermediate arrays.- Why MAD is robust: MAD uses medians (median of values and median of absolute deviations), so a small number of extreme values have little effect. Unlike mean/std, a single extreme value won't pull the median much, making MAD stable for heavy-tailed or contaminated data.- Edge cases: non-numeric inputs (cast to float), NaNs preserved (not flagged), MAD==0 handled to avoid division by zero (could alternatively switch to IQR-based detection). Alternatives: z-score (mean/std) — less robust; IQR rule — also robust but measures spread differently.

MediumTechnical

66 practiced

Give a concrete example where applying a log transform to a skewed numeric feature changes coefficient magnitudes and p-values in linear regression. Explain why the interpretability and inference change and how you would validate that the transform improves downstream predictive performance and assumptions (e.g., residual normality, heteroskedasticity).

Sample Answer

Example:Suppose I predict house price (price) with lot size (lot_sqft). lot_sqft is highly right-skewed (many small lots, few huge ones). Fit two OLS models:

Model A (raw):price = β0 + β1 * lot_sqft + εEstimated β1 = 0.00045 (p = 0.04). A few very large lots drive both coefficient magnitude and residual variance.

Model B (log transform):price = β0 + β1_log * log(lot_sqft) + εEstimated β1_log = 28,000 (p < 0.001). Coefficient magnitude and significance changed: instead of tiny per-square-foot effect, β1_log represents expected dollar change for a multiplicative (log) change in lot size — roughly, a 1% increase in lot size ≈ 280 dollar increase (approx because Δprice ≈ β1_log * Δlog(lot_sqft)).

Why changes occur:- Log reduces right skew and compresses outliers, so extreme lots no longer dominate estimates.- The transform often linearizes multiplicative relationships (price grows roughly proportionally to lot size), improving model fit.- Variance of residuals can stabilize (heteroskedasticity reduced), which tightens standard errors and thus p-values.

How interpretability changes:- Raw coefficient: dollars per square foot — intuitive but sensitive to scale and outliers.- Log predictor: effect per percent change in lot size (elasticity-like interpretation). If both response and predictor are logged, coefficients become elasticities (percent change in price per percent change in lot size).

How to validate the transform improves inference and prediction:1. Residual diagnostics (compare A vs B): - Q–Q plot and Shapiro-Wilk for normality. - Plot residuals vs fitted: check heteroskedasticity visually. - Breusch–Pagan or White test for heteroskedasticity. - Check influential points (Cook’s distance); log typically reduces extreme influence.

2. Model fit and parsimony: - Compare adjusted R² and AIC/BIC (higher adj. R² or lower AIC favors the transform). - Inspect coefficient standard errors and p-values for stability.

3. Predictive performance (out-of-sample): - k-fold CV comparing RMSE/MAE and calibration. - If using log-transformed response, back-transform predictions (apply bias correction, e.g., adding half the residual variance when exponentiating) and compare metrics on original scale. - Compare prediction intervals coverage.

4. Sensitivity checks: - Try alternatives (sqrt, Box–Cox) and see which best meets assumptions. - Re-run with robust standard errors to confirm inference robustness.

Conclusion:A log transform can reduce skew, stabilize variance, and better align the model with the true generative relationship; diagnostics (residual tests, CV, information criteria) and sensitivity checks quantify whether the transform improves interpretability, inference, and predictive performance.

MediumTechnical

71 practiced

List fairness metrics applicable to binary classification (demographic parity, equalized odds, predictive parity, calibration per group). For each metric explain how it's computed, what it measures, and one scenario where optimizing it could hurt another fairness criterion.

Sample Answer

Demographic parity (statistical parity)- Computation: P(Ŷ=1 | A=a) equal across protected groups a (compare rates or difference/ratio).- Measures: Whether positive prediction rates are the same regardless of group membership (focuses on group-level selection rates).- Conflict example: Enforcing parity can force raising positives for a low-base-rate group, harming equalized odds (increasing false positives) and hurting utility or individual fairness.

Equalized odds- Computation: For each true label y∈{0,1}, require P(Ŷ=1 | A=a, Y=y) equal across groups (equal TPR and equal FPR).- Measures: Whether the classifier’s error rates are the same across groups (fairness of outcomes conditional on ground truth).- Conflict example: Achieving equalized odds may break predictive parity/calibration — different predictive values across groups because identical error rates don't guarantee equal PPV.

Predictive parity (positive predictive value parity)- Computation: P(Y=1 | Ŷ=1, A=a) equal across groups (compare PPV or NPV).- Measures: Whether a positive prediction has the same reliability for all groups.- Conflict example: Optimizing PPV parity typically conflicts with equalized odds when base rates P(Y=1|A) differ — you can’t generally have both unless base rates and error rates align.

Calibration per group (groupwise calibration)- Computation: For scores s, P(Y=1 | score=s, A=a) = s for all groups (or binned: observed positive rate within score bin equals average score).- Measures: Whether predicted probabilities reflect true likelihoods equally across groups.- Conflict example: Group calibration can conflict with equalized odds and demographic parity — calibrated scores plus equalized odds imply constraints that are often impossible when prevalence differs, forcing trade-offs between well-calibrated probabilities and equal error rates.

Takeaway: These metrics capture different notions (selection rates, error parity, predictive reliability, probabilistic correctness). With differing base rates and finite classifiers, satisfying all simultaneously is usually impossible; pick metrics aligned to legal, ethical, and application-specific goals and be explicit about trade-offs.

MediumTechnical

125 practiced

Describe how you'd implement data quality checks using Great Expectations for a column 'price' with the rules: at least 95% non-null, values between 0 and 1000, and expected top-3 categories for a categorical column 'tier' being ['A','B','C']. Provide a sample expectation or pseudo-code and state assumptions about sample sizes.

Sample Answer

Approach: create a Great Expectations suite with three expectations: (1) percent non-null for price >= 0.95, (2) numeric range between 0 and 1000 for price (allowing some outliers according to tolerance), (3) expected top-3 categories for tier are ['A','B','C'] (validate by top_k and proportions). I assume batches are reasonably large (n >= 1000) so percentages are stable; for small batches use absolute counts or lower confidence.

Sample implementation (pythonic pseudo/real GE calls):

python

from great_expectations.core.expectation_configuration import ExpectationConfiguration
from great_expectations.dataset import PandasDataset

# Given a GE dataset (PandasDataset or batch)
ds = PandasDataset(df)

# 1) At least 95% non-null for 'price'
ds.expect_column_values_to_not_be_null("price", mostly=0.95)

# 2) Values between 0 and 1000 (allow a tiny fraction outside if needed)
ds.expect_column_values_to_be_between("price", min_value=0, max_value=1000, mostly=0.995)

# 3) Top-3 categories expected to be A,B,C
# Validate that the top_k categories equal the expected list (order-insensitive).
res = ds.expect_table_column_value_counts_to_be_in_set(
    column="tier",
    value_set=["A","B","C"],
    mostly=0.90, # at least 90% of counts are within the set
    result_format={"result_format":"SUMMARY"}
)

# Alternatively assert top_k categories exactly:
ds.expect_column_distinct_values_to_be_in_set("tier", ["A","B","C"], mostly=0.75)

Reasoning:- Use mostly to allow noise/outliers while enforcing thresholds.- For categorical top-3, checking that a high share of values belong to A/B/C is pragmatic; exact top_k equality can be brittle.Assumptions:- Batch size >= 1000 for percentage stability; if <100, prefer absolute counts (e.g., expect_table_row_count_to_be_between) or widen thresholds.Edge cases:- Nulls encoded as sentinel strings — normalize first.- Skewed distributions; consider monitoring drift over time rather than rigid equality.

Unlock Full Question Bank

Get access to hundreds of Data Quality and Bias interview questions and detailed answers.

Join thousands of developers preparing for their dream job.