Exploratory Data Analysis and Data Quality Questions

Direct answer

Treat this as an evidence-gathering exercise for each flagged point, not a blanket rule: check whether the value is physically plausible, whether it correlates with a known legitimate cause (a VIP account, a bulk order, a specific promotion), and whether removing or capping it would change a downstream decision. Only remove a point once you have a specific reason to distrust it, not just because it's far from the rest.

A decision process

1. Check plausibility first. Is the value even possible given the business (a negative quantity, an age of 300)? If so, it's very likely an error, not a legitimate extreme value.
2. Look for corroborating signal. Does the account have a history of similarly large orders? Is there a promotion or bulk-purchase flag active that explains it? A genuine VIP purchase usually has supporting evidence elsewhere in the row or the customer's history; a data-entry error usually doesn't.
3. Consider the downstream use. If you're reporting a sum (total revenue), an error needs fixing but a legitimate large value should stay in. If you're reporting a "typical" value (median order size), a genuine extreme value can often stay in the data untouched, since the median won't be swayed by it anyway.
4. Choose the least destructive fix that solves the actual problem. Removing a row throws away information permanently; winsorizing (capping at a threshold) keeps the row but limits its influence; a log transform reduces the influence of all large values proportionally rather than singling one out. Reach for the most surgical option that addresses what's actually wrong.

Worked example

Say a $50,000 order shows up in a dataset where the 99th percentile is $1,200. Checking the account: it belongs to a corporate customer with three other orders over $30,000 in the past year, and the order line items list 500 units of the same product, consistent with a bulk purchase. That's a legitimate outlier, kept as-is for a sum, and probably fine to leave untouched for a median too since it won't move it much. Contrast that with a second $50,000 order from a first-time individual customer buying a single unit of a $12 product: no supporting evidence, and a quick check shows the quantity field was likely fat-fingered as 4,000 instead of 4. That one gets corrected (or excluded and flagged), not because it's large, but because there's a specific, checkable reason to distrust it.

Trade-offs and pitfalls

The temptation to "just remove outliers because they skew the numbers" is a real trap: skewing the numbers is sometimes the entire point, if the numbers are genuinely skewed. Document whatever decision you make and why, since a silent removal is invisible to anyone auditing the analysis later, and a wrong call here (removing real revenue, or keeping a fat-fingered order) directly changes a business number.

Direct answer

Investigate a near-constant column (95% one value, 5% something else) by first checking whether the rare non-null values are genuinely informative or just noise, and whether the column's near-constant nature is a real property of the world or an artifact of how the data was collected. If the rare values correlate meaningfully with anything else you care about, the column is worth keeping, likely as a flag ("is this the common value or not") rather than as a raw feature; if the rare values look like noise or errors, the column may be safe to drop.

What would change the answer

Check whether the 5% of non-default values cluster around anything specific (a particular time period, a particular customer segment, a particular data source), which would suggest they're a real, meaningful minority case rather than noise. Also check whether the dominant value looks like a genuine business state (say, "active" status for most rows) versus a suspicious default that might indicate a field nobody actually fills in reliably (like an optional form field left at its pre-filled default 95% of the time). The two scenarios call for very different conclusions even though the raw statistics look identical.

Worked example

A subscription_tier column is "free" for 95% of rows and something else for the remaining 5%. Checking whether that 5% correlates with anything: it turns out those users have dramatically higher engagement and retention, meaning the column, despite being heavily skewed, is carrying real, valuable signal about a small, genuinely different population. Contrast that with a referral_source field that's the literal string "unknown" for 95% of rows and a mix of real values for the rest: checking whether "unknown" correlates with a specific signup date range reveals it was simply never collected before a certain date, meaning the near-constant value here is a data-collection artifact, not a meaningful state.

Trade-offs and pitfalls

Don't drop a near-constant column reflexively just because it looks like "not much information": a rare-but-meaningful minority value can be one of the most useful signals in the whole dataset, precisely because it's rare and distinguishes a small group from everyone else.

Direct answer

Run a specific checklist: compare feature distributions across the train, validation, and test splits (a feature that looks dramatically different across splits is suspicious); look for duplicate or near-duplicate rows leaking across those splits; verify time-based splits are actually respecting chronology rather than mixing future and past rows; and check whether a single feature, on its own, can predict the target suspiciously well, which is a classic signature of a feature that's secretly derived from or entangled with the outcome.

Why each check catches a different flavor of leakage

Comparing distributions across splits catches accidental contamination, like the same customer's rows ending up in both train and test after an ID-based split that didn't account for a customer having multiple rows. Checking for duplicate rows across splits catches the same underlying problem more directly. Verifying time-based splits catches TEMPORAL leakage, where a supposedly historical training set actually contains information from after the point you're pretending to make a prediction at. Testing whether a single feature alone achieves suspiciously high predictive power is often the fastest way to catch schema-derived leakage, since a feature that's mechanically tied to the answer (or only exists once the answer is already known) tends to dominate a model far more than any single legitimate feature realistically would.

Worked example

A churn model shows unexpectedly high held-out performance. Running the checklist: feature distributions look consistent across splits, ruling out obvious split contamination; but a single feature, days_between_last_login_and_churn, turns out to be defined ONLY for customers who already churned (it's undefined or null for everyone who didn't), meaning the feature mechanically encodes the answer rather than predicting it. Training a model on that one feature alone reproduces almost the entire "surprising" performance, confirming it as the leak; removing it and re-checking the model's performance drops to a much more plausible, and much more trustworthy, level.

Trade-offs and pitfalls

A checklist this systematic takes real time to run properly, but skipping it and just accepting a suspiciously strong result is how leakage makes it all the way into a shipped model that then quietly underperforms (or fails outright) once it meets data where the leaked shortcut no longer exists.

Direct answer

A histogram is the default first look at a numeric distribution's overall shape; a boxplot compresses that into a five-number summary plus outlier flags and is best for quickly comparing many groups side by side; a violin plot adds a smoothed density outline around the boxplot, recovering the shape information a boxplot alone hides, at the cost of needing more data to look stable; a KDE (kernel density estimate) shows a smooth version of the histogram's shape without being sensitive to bin-width choices, but can smooth away genuinely sharp features like a hard floor or ceiling in the data.

What each emphasizes

Histograms show gaps, multiple modes, and the exact shape, but the picture changes with bin width, sometimes substantially. Boxplots are compact and comparable across many categories at once (ten boxplots side by side is readable; ten histograms side by side usually isn't), but they can make a genuinely bimodal distribution look like an ordinary unimodal one with some spread. Violin plots recover that bimodality by showing the density shape directly, which matters when you specifically suspect two or more distinct subpopulations hiding inside one column. KDEs are the smoothest and most visually appealing for a single distribution, but the smoothing bandwidth is a hidden parameter that can understate or overstate how sharp a real feature is.

Worked example

Exploring session_duration for two user segments: a boxplot side by side for "free" and "paid" users quickly shows the paid segment has a higher median and a longer whisker. But a violin plot for the free segment on its own reveals two distinct humps, a very short-session group and a long-session group, that the boxplot's single box completely hid. That distinction (two subpopulations rather than one spread-out population) is exactly the kind of finding that changes what you investigate next.

Trade-offs and pitfalls

For a technical audience comparing many groups at once, boxplots (or violin plots if bimodality is suspected) scale better than a wall of histograms. For a single distribution you're exploring in depth yourself, a histogram alongside a KDE is usually the most informative combination. For a non-technical audience, a single clean histogram or a simple boxplot is usually more legible than a violin plot, whose density-shape encoding takes explanation.

Direct answer

Exploratory analysis is open-ended pattern-finding on data you already have, meant to generate hypotheses and build intuition; confirmatory (inferential) analysis is a pre-specified test of a hypothesis, ideally on fresh data, meant to decide whether an observed pattern is real or noise. The distinction matters because a pattern discovered through open-ended exploration hasn't been tested against anything yet: if you go looking hard enough across many slices of the same data, you WILL find something that looks interesting purely by chance, and treating that discovery as already-proven is how EDA quietly turns into unintentional p-hacking.

The transition from one to the other

EDA should end with a specific, falsifiable hypothesis ("weekend signups convert at a lower rate than weekday signups"), not a conclusion. Confirmatory analysis then tests that specific hypothesis, ideally against data or a time window you didn't use to generate the hypothesis in the first place, using a pre-registered test rather than whichever test happens to come back significant. Descriptive statistics (what happened) and inferential statistics (what we can conclude about the broader pattern) map onto this same boundary: summarizing last quarter's churn rate is descriptive, while estimating whether a new pricing tier will reduce churn going forward requires inference.

Worked example

While exploring an engagement drop, you notice active users dipped 8% the week a UI change shipped. That's an EDA finding, a hypothesis, not a proof: the same week also had a holiday, a competitor promotion, and normal week-to-week noise. Confirmatory analysis means designing a specific test (a difference-in-differences comparison against a similar prior period, or ideally a proper A/B test on the UI change itself) and pre-specifying what result would count as support versus refutation, rather than continuing to slice the exploratory data until you find a cut that confirms your initial hunch.

Trade-offs and pitfalls

The most common failure is presenting an EDA finding with the confidence of a confirmatory result, skipping the formal test entirely because the pattern "looks obviously real." The complementary failure is over-formalizing every EDA finding with a full statistical test when a quick sanity check would do, which slows down the exploratory phase without adding real rigor if the underlying hypothesis was going to be tested properly downstream anyway.