Exploratory Data Analysis Questions

Exploratory Data Analysis is the systematic process of investigating and validating a dataset to understand its structure, content, and quality before modelling or reporting. Core activities include examining schema and data types, computing descriptive statistics such as counts, means, medians, standard deviations and quartiles, and measuring class balance and unique value counts. It covers distribution analysis, outlier detection, correlation and relationship exploration, and trend or seasonality checks for time series. Data validation and quality checks include identifying missing values, anomalies, inconsistent encodings, duplicates, and other data integrity issues. Practical techniques span SQL profiling and aggregation queries using GROUP BY, COUNT and DISTINCT; interactive data exploration with pandas and similar libraries; and visualization with histograms, box plots, scatter plots, heatmaps and time series charts to reveal patterns and issues. The process also includes feature summary and basic metric computation, sampling strategies, forming and documenting hypotheses, and recommending cleaning or transformation steps. Good Exploratory Data Analysis produces a clear record of findings, assumptions to validate, and next steps for cleaning, feature engineering, or modelling.

HardSystem Design

62 practiced

Design a production-ready data drift monitoring system for an AI model: specify which metrics to track (PSI, KS test, classifier-based drift score), sampling cadence, storage of historical distributions and summaries, alerting thresholds, automatic EDA checks to run on drifted features, and how to trigger retraining, model rollbacks, or human review.

Sample Answer

Requirements and constraints:- Monitor feature/input distribution and model-output drift in production for latency-sensitive neural models; support per-feature and multivariate drift, lineage, audit logs, and automated remediation with human-in-loop.- Scale to millions of examples/day, retention of summaries for 12+ months, and low compute overhead.

High-level architecture:- Ingest (stream) → Feature extractor & sampler → Metrics engine (batch & online) → Drift store (time-series + object store) → Alerting & Runbook service → Automation (retrain/orchestrator) + Dashboard

Metrics to track:- Univariate: - Population Stability Index (PSI) per numeric/binned feature (daily / per-window) - Kolmogorov–Smirnov (KS) statistic for continuous features - Chi-squared for categorical distributions- Multivariate / model-aware: - Classifier-based drift score: train a domain classifier (prod vs ref) and report AUC; higher AUC → stronger drift - Prediction distribution shifts (mean, std, tail percentiles) - Label-aware metrics when labels available: target distribution change, label shift correction (e.g., BBSE)- Data quality: missingness, outlier rate, schema changes, cardinality changes- Business metrics: model accuracy, precision/recall, downstream KPIs

Sampling cadence and windows:- Online micro-batch: sample every 5–15 min for quick detection (low-cost sketches)- Aggregated windows: hourly summaries, daily full-window checks, 7/30-day baselines- Reference baselines: rolling baseline (last N days), stable baseline (production-chosen checkpoint), and training set snapshots

Storage of distributions & summaries:- Time-series DB (e.g., Prometheus/Influx/ClickHouse) for metric traces and alerts- Object storage (S3) for histograms, quantile sketches (t-digest), and serialized baseline distributions- Feature registry / metadata store (Feast/MLMD) for lineage and schema- Retain raw aggregated histograms and t-digests for 12 months; retain full sample snapshots for 90 days

Alerting thresholds & logic:- Multi-tier alerts: - Warning: PSI > 0.1 or KS p-value < 0.05 or classifier AUC > 0.6 - Critical: PSI > 0.25 or KS p-value < 0.01 or classifier AUC > 0.75 or sudden schema change- Use combined signals: trigger only if >=2 metrics exceed thresholds OR metric persists >24 hours to reduce noise- Include adaptive thresholds (seasonality-aware) and anomaly detection on metric time-series

Automatic EDA checks when drift detected:- Run automated EDA job for drifted features: - Plot baseline vs current histograms, CDFs, and t-digest percentiles - Correlation heatmap before/after; mutual information change vs target - Check missingness patterns, new categories, and cardinality growth - Feature importance delta: re-run SHAP/feature attribution on a recent labeled sample to see importance shifts - Root-cause heuristics: upstream source change, deployment/version change, upstream schema mods- Generate human-readable drift report with suggested actions and confidence scores; attach sample records and provenance.

Retraining, rollback, and human review policy:- Automatic retrain (safe mode): - Preconditions: labeled validation data available and retrain validation passes (>= baseline metrics), drift persistent > X days, and PR-style gating (automated tests). - Pipeline: spawn retrain job (same training pipeline), run automated validation (unit tests, fairness/regulatory checks, performance on holdout), staging A/B test for >=K requests or time window. - If new model outperforms and passes checks → promote.- Human review required: - If drift affects high-risk features or business KPI drop > threshold, or if automatic validation fails, create ticket for data scientist review with EDA report.- Immediate rollback: - If live model metrics degrade beyond emergency threshold (e.g., accuracy drop >10% or critical business loss), trigger automated rollback to last known-good model and notify on-call.- Audit & safety: - All automated actions logged; approvals for retraining/promotions via CI/CD gating; can require 1–2 approvers depending on risk.

Operational considerations:- Rate-limit metric computations and use sketches (t-digest, count-min) for large cardinalities.- Backfill capability for delayed labels.- Monitor monitoring system health (watchdog alerts).- Provide dashboards with per-feature drilldowns, runbook links, and simulation tools for proposed thresholds.

This design balances sensitivity and noise reduction, provides explainability for root cause, and enforces safe automated remediation with human oversight for high-risk decisions.

EasyTechnical

58 practiced

As an AI Engineer, propose a concise EDA report template you would deliver after initial dataset exploration. Include sections (overview, schema summary, missingness, outliers, class balance, label-quality, visualizations, hypotheses, recommended cleaning steps), what artifacts to attach (notebooks, CSV summaries, images), and versioning/metadata (data hash, sample seed) to ensure reproducibility for downstream engineers and stakeholders.

Sample Answer

Overview- One-paragraph summary: dataset origin, date ingested, objective (task type: classification/regression/segmentation), sample size, high-level takeaways.

Schema summary- Table of columns: name, type (numeric/categorical/text/image), example values, cardinality, semantic notes (e.g., timestamp timezone).

Missingness- Missing count/percent per column, missingness pattern (MCAR/MAR/NMAR assessment), columns to impute vs. drop, impact on labels.

Outliers- Method used (IQR/Z-score/robust MAD), flagged rows count, suggested handling per column (cap/remove/transform).

Class balance / target distribution- Class counts, percent, imbalance ratio, per-class feature coverage, suggested resampling if needed.

Label quality- Label leakage checks, consistency checks, label noise estimate (confusion on repeated samples), recommended relabeling or human review samples.

Visualizations (attach images)- Histogram/density per numeric feature (selected), boxplots for outliers, bar plots for categorical frequency, correlation heatmap, t-SNE/UMAP sample of embeddings, target vs-feature plots.

Hypotheses- 3–5 testable hypotheses (feature predictive power, temporal drift, label bias), prioritized.

Recommended cleaning steps (concrete)- Ordered checklist with code snippets references: dedupe, impute strategy, normalization, encoding, date parsing, text cleaning, train/val split method (time-based if needed).

Artifacts to attach- Executable notebook(s) with reproducible cells- CSV summaries: schema_summary.csv, missingness_report.csv, outlier_report.csv, label_audit.csv- Visualization images (.png/.svg)- Sample of raw and cleaned rows (sample_seeded.csv)

Versioning / metadata for reproducibility- Data hash (SHA256) of source file(s), dataset version tag, extraction SQL/query, sample seed value, environment spec (python packages + versions or conda env YAML), notebook commit SHA, run timestamp.

Stakeholder call-to-action- 2–3 clear next steps for stakeholders: approve cleaning plan, allocate label-review budget for N samples, or confirm evaluation metric to prioritize.

HardTechnical

82 practiced

Propose metrics and visualizations to quantify annotator uncertainty and disagreement in a multi-label annotation task: per-item entropy across annotators, annotator confusion matrices, label co-occurrence heatmaps, and a dashboard that surfaces items with highest disagreement. Discuss how EDA results should influence model training choices (soft labels, label smoothing, multiple supervisory signals).

Sample Answer

Metrics and visualizations- Per-item entropy (multi-label): for item i, compute annotator empirical probability p_j for each label j (fraction of annotators who marked label j). Item entropy: H(i) = - sum_j [ p_j log p_j + (1-p_j) log(1-p_j) ] (sum over labels independently, or compute joint entropy for label vectors). Visuals: histogram + violin of H(i); cumulative percentiles; time series if annotation rounds evolve.- Label-level disagreement: for each label j compute disagreement rate = 1 - mean agreement (e.g., mean pairwise Cohen’s kappa or Krippendorff’s alpha across annotators for that label). Visuals: bar chart with confidence intervals.- Annotator confusion matrices: per-annotator vs majority / vs each other: normalized confusion matrices (precision/recall per label) and a matrix of pairwise disagreement scores (1 - pairwise agreement). Visual: clustered heatmap to reveal annotator groups or systematic bias.- Label co-occurrence heatmap: show empirical co-occurrence frequencies and conditional probabilities P(label B | label A). Visual: symmetric and asymmetric heatmaps, plus UMAP/t-SNE of items colored by high-disagreement clusters.- Item-level dashboard: sortable table of items ranked by entropy/disagreement, showing raw annotations, timestamps, annotator IDs, and model predictions; filters by label, domain, annotator, and disagreement threshold. Include sampling widgets for spot-checking.- Additional metrics: Jaccard similarity distribution across annotator pairs, per-item majority confidence (fraction in majority), calibration plots of annotator consensus vs model confidence.

How EDA should influence training choices- Soft labels / probabilistic targets: if many items show intermediate p_j (e.g., 0.3–0.7), replace hard 0/1 with annotator-frequency vectors as training targets and train with KL-divergence or BCE against probabilities. This preserves uncertainty and prevents overfitting to noisy consensus.- Label smoothing vs learned uncertainty: label smoothing helps label noise but is uniform; prefer empirical smoothing using annotator frequencies when available. Use label smoothing only for labels with high annotator calibration.- Multiple supervisory signals: when annotators show systematic biases, consider multi-head models: one head predicts consensus distribution; auxiliary heads predict annotator-specific behavior or propensity-to-label. Multi-task losses can disentangle true signal vs annotator bias.- Loss choices: weight examples by disagreement—use higher-temperature KL or focal loss for high-disagreement items to prevent dominating gradients, or conversely upweight them for model to learn ambiguous boundary cases depending on objective (robustness vs precision).- Calibration & thresholding: use EDA to set per-label thresholds (precision-recall trade-offs) and to calibrate probabilistic outputs (temperature scaling), especially where co-occurrence indicates conditional label dependencies.- Active learning & relabeling: surface top-disagreement items for expert adjudication or additional annotation; use adjudicated items to create high-quality gold set and to estimate annotator reliabilities.- Evaluation: evaluate against probabilistic targets (e.g., expected log-likelihood, KL) and report metrics stratified by disagreement bins. Avoid reporting only accuracy on majority labels.

Practical notes- If disagreement clusters by annotator, model annotator ID or use annotator embeddings during training.- For rare labels with high disagreement, prefer human review and conservative deployment thresholds.- Document decisions from EDA: which labels use soft targets, which needed adjudication, and how thresholds were chosen.

HardTechnical

73 practiced

Explain how to estimate mutual information (MI) and conditional mutual information between features and a target during EDA, particularly for high-cardinality categorical features. Cover discrete plug-in estimators, k-NN estimators for continuous variables, and practical strategies (smoothing, grouping rare levels) to make MI estimation robust and computationally feasible.

Sample Answer

Start by defining MI: I(X;Y)=E[log p(X,Y)/(p(X)p(Y))], and CMI: I(X;Y|Z)=E[log p(X,Y|Z)/(p(X|Z)p(Y|Z))]. Estimation reduces to estimating (conditional) densities or discrete probabilities.

1) Discrete plug-in estimators- Compute empirical joint/ marginal counts, convert to probabilities, plug into MI formula. Fast and intuitive.- Problems: severe upward bias when counts are small (especially high-cardinality features). Use bias correction (Miller–Madow) or Bayesian smoothing (Laplace/Dirichlet priors) to stabilize estimates.- For CMI, estimate joint counts over (X,Y,Z) — dimensionality explodes; apply conditioning only on low-cardinality Z or aggregate Z levels.

2) k-NN / continuous estimators- Use k-NN entropy estimators (Kraskov et al.) that estimate differential entropies via neighbor distances and combine to get MI. Works well for continuous variables, avoids explicit density estimation.- For mixed discrete-continuous, discretize continuous or use mixed estimators (e.g., add small noise to discrete levels or use copula transforms).

3) Practical strategies for high-cardinality categorical features- Group rare levels: lump infrequent categories into “Other” or hierarchical buckets based on domain knowledge or target statistics to reduce sparsity.- Target-driven grouping: merge levels with similar P(Y|level) using clustering (e.g., hierarchical clustering on level-wise target rate) — preserves predictive signal while lowering cardinality.- Hashing or feature hashing to limit dimensionality, but be aware of collisions biasing MI downward.- Regularized target-encoding (with smoothing toward global mean) provides robust surrogate for MI ranking; cross-validate to avoid leakage.- Apply Laplace smoothing or Bayesian Dirichlet priors to counts before plug-in MI.- Subsample or use stratified sampling to compute MI at scale; parallelize count aggregations.- Permutation-based significance testing: compare observed MI to null MI from permuted target to gauge significance and correct for spurious MI from many levels.

4) Computational & evaluation notes- Normalize MI (e.g., divide by min(H(X),H(Y)) or use Mutual Information Ratio) for comparability across features.- For CMI, limit conditioning variables or use conditional independence tests with low-dimensional projections (e.g., partial MI via residuals from predictive models).- Validate with simple predictive models:MI is a filter; confirm that high-MI features improve downstream model performance; guard against leakage.

Summary: Use plug-in with smoothing and bias correction for categorical data, k-NN/Kraskov for continuous, and combine practical level grouping, target-driven aggregation, regularized encoding, and permutation tests to make MI/CMI estimation robust and scalable for high-cardinality features.

EasyTechnical

104 practiced

You have user activity timestamps across two years for millions of users. Describe concrete aggregation queries or concise pandas code and visualizations you'd run to detect seasonality, daily/weekly patterns, and data gaps. Include how you'd handle timezone normalization, daylight savings issues, and sparse user activity while summarizing results to stakeholders.

Sample Answer

Approach: normalize timestamps to a single timezone, bucket events at multiple granularities (hour/day/week/month), compute user-level and global aggregates, plot heatmaps/time series and report data gaps and sparsity metrics.

Code (pandas + seaborn):

python

import pandas as pd
import numpy as np
import seaborn as sns
import matplotlib.pyplot as plt

# df: columns ['user_id','ts'] where ts is timezone-aware or naive UTC
df['ts'] = pd.to_datetime(df['ts'], utc=True)            # parse as UTC
df['local'] = df['ts'].dt.tz_convert('America/Los_Angeles')  # normalize to chosen TZ

# features
df['date'] = df['local'].dt.date
df['hour'] = df['local'].dt.hour
df['weekday'] = df['local'].dt.day_name()
df['week'] = df['local'].dt.to_period('W').apply(lambda r: r.start_time)

# global daily counts
daily = df.groupby('date').size().rename('events').reset_index()
daily['date'] = pd.to_datetime(daily['date'])
sns.lineplot(data=daily, x='date', y='events'); plt.title('Daily events')

# weekly-seasonality: average by weekday
by_wd = df.groupby(['weekday','hour']).size().unstack(fill_value=0)
# order weekdays
order = ['Monday','Tuesday','Wednesday','Thursday','Friday','Saturday','Sunday']
by_wd = by_wd.reindex(order)
sns.heatmap(by_wd, cmap='viridis'); plt.title('Weekday x Hour heatmap')

# monthly seasonality
monthly = df.set_index('local').resample('M').size()
monthly.plot(); plt.title('Monthly totals')

Key points:- Timezones/DST: store UTC; convert to a business-relevant tz only for aggregation/visuals. Use pandas tz-aware datetimes to handle DST (ambiguous/nonexistent times) via errors='coerce' or ambiguous parameters.- Data gaps: compute expected timeline and left-join to counts to find missing days/weeks:

python

full = pd.date_range(daily.date.min(), daily.date.max(), freq='D')
missing = set(full.date) - set(daily['date'])

- Sparse users: compute per-user active days and pareto (top users vs long tail). Use deciles/percentiles:

python

user_days = df.groupby('user_id')['date'].nunique()
user_days.describe(percentiles=[.5,.9,.99])

Summarizing to stakeholders:- One-slide summary: key seasonality findings (peak hours, weekdays/months), data gaps (date ranges with no data and likely causes), and sparsity (percent of users with >X events). Provide actionable recommendations: adjust model training windows to avoid DST anomalies, downsample heavy users, impute or exclude gap periods, and monitor ingest for recurring gaps. Include visuals (line + heatmap + histogram) and simple numeric bullet points (top-3 insights).

Unlock Full Question Bank

Get access to hundreds of Exploratory Data Analysis interview questions and detailed answers.

Join thousands of developers preparing for their dream job.