InterviewStack.io LogoInterviewStack.io
Interview Prep10 min read

Data Scientist Class Imbalance Interview: The 0.6% Fraud Trap

A 0.6% fraud rate, delayed labels and two new countries: walk a mid-level Data Scientist class imbalance interview turn by turn and see where points leak.

IT
InterviewStack TeamData
|

A 0.6% Fraud Rate Makes Every Data Prep Shortcut Expensive

Picture a mid-level Data Scientist interview where nobody asks you to write a single line of model code. You get a fraud scenario with a 0.6% positive rate, labels that arrive late, and two new countries that behave differently. The candidates who lose points here usually know the techniques. They lose them on the order of decisions, and they do not see it happening.

This walkthrough is a simulated mock interview on Data Scientist roles, built from one real AI interview blueprint for Mid-level (2-5 years). The answers below are dramatized and illustrative: "a common answer" is a pattern, not a transcript of anyone. The scenario is generic, not any company's actual question.

Key Findings

  • The interview runs 30 minutes across 3 phases: framing (0-8), training data design (8-20), and shift and cold start (20-30).
  • Scoring is out of 100: Interviewer Objectives Alignment 30, Level-Specific Expectations 30, Technical Proficiency 20, Communication and Problem Solving 20.
  • Technical Proficiency is only 20 of 100 points, so framing and trade-offs carry 60 points between them.
  • The scenario has a 0.6% fraud rate, so accuracy tells you almost nothing.
  • Phase 2 is the longest at 12 minutes and holds 6 checklist items, including separating probability from threshold.
  • Features come from the first 7 days after signup, and the business wants predictions earlier than that.
  • Two new countries launched 3 months ago, and the newest has very few confirmed fraud labels.

Interviewer scoring weights across the four rubric dimensions, 100 points total

Sixty of the 100 points reward framing and level-appropriate trade-offs, not tool trivia.

What Is the Data Scientist Class Imbalance Interview Really Testing?

The question is open-ended on purpose. The interviewer wants a prioritized plan, not a tour of SMOTE variants.

The interview question

You are supporting a trust and safety team for a large consumer platform. The team wants a model that predicts whether a newly created seller account is likely to become fraudulent within 30 days of signup, so high-risk accounts can be reviewed early.

Only about 0.6% of seller accounts are eventually labeled fraudulent. Labels often arrive late because many cases are confirmed only after user reports or chargebacks. The product launched in two new countries 3 months ago and behavior there looks different. Several key features come from the first 7 days after signup, but the business wants predictions as early as possible. The newest country has very few confirmed fraud labels. Missing fraud loses money, but false positives add manual review burden and hurt legitimate sellers.

How would you prepare the training data for this model so that it can learn something reliable and be usable in production?

Underneath, the interviewer is probing five things: label quality, leakage, how you build train, validation and test sets, imbalance handling, and what you can honestly claim about a market with almost no labels. Here are four follow-ups, and the quiet point losses behind them.

The Walkthrough: Four Turns, Four Quiet Point Losses

Turn 1: Define the label

Interviewer: "How would you define positive and negative labels given that fraud confirmation is delayed and some accounts have not had enough time to mature yet?"

COMMON MISTAKE
Marcus labels every account with no fraud report as a negative, so 30-day-old signups become confirmed clean sellers. That misses the phase 1 checklist item that immature accounts are not confirmed negatives, and it quietly costs Interviewer Objectives Alignment.
STRONGER MOVE
Name the prediction time, an observation window, and a future outcome window. Add a label maturity cutoff, then exclude or hold out accounts younger than the cutoff instead of calling them negatives.
### Turn 2: Leakage from day-7 features

Interviewer: "What would you do to avoid leakage when features come from the first 7 days of activity but the business wants earlier predictions?"

COMMON MISTAKE
Marcus keeps the day-7 activity features and says the model will simply run later in production. That ignores feature consistency between training and serving, and it skips the mid-level expectation to reason about feature availability at inference time (Level-Specific Expectations).
STRONGER MOVE
Pick the decision point first, then rebuild features strictly from data that exists at that moment. Also check for post-prediction events such as investigations or reports sneaking into the feature set.
### Turn 3: Weights, resampling, or thresholds

Interviewer: "How would you choose between resampling, class weighting, and decision-threshold tuning for this problem?"

COMMON MISTAKE
Marcus oversamples the fraud class until it is 50/50 and reports ROC-AUC as the win. That blends probability estimation with the decision rule and ignores review capacity, which costs the checklist item on separating probability from threshold and rare-event metrics (Technical Proficiency).
STRONGER MOVE
Treat imbalance handling as a training choice (class weights, limited resampling) and the threshold as a business choice tied to reviewer capacity. Judge with precision-recall, or recall at fixed precision and alert volume.
### Turn 4: Strong offline, weak abroad

Interviewer: "Suppose the model performs well offline overall but poorly in the two newest countries. How would you diagnose whether this is class imbalance, distribution shift, or a labeling problem?"

COMMON MISTAKE
Marcus jumps straight to adding more fraud examples from the new countries. With no comparison of feature and label distributions across countries and time, that skips the shift diagnosis item and costs Communication and Problem Solving.
STRONGER MOVE
Diagnose before fixing: slice validation by country and time, compare feature and label distributions, and check whether late labels are making new markets look cleaner than they are. Then name a fallback such as a global model with country features, or rules plus model.

Why Isn't Reading This Enough?

Spotting these mistakes on the page took seconds. Avoiding them live is different: the follow-ups are unscripted, the clock is running, and a confident wrong first answer anchors everything after it. The gap between recognizing a good answer and producing one under pressure only closes with reps.

The Blueprint a Strong Candidate Hits

The 30-minute interview paced into three phases

This is the blueprint a strong candidate hits, and the exact structure the AI mock interview tracks you against in real time. Read each checklist item as a question you should be able to answer unprompted.

Blueprinta strong 30-minute interview, phase by phase
1
Problem framing and label construction 0-8
  • ✓Clarifies prediction time: signup, day 1, day 3, or day 7, and notes that this choice changes usable features
  • ✓Defines an observation window and a future fraud outcome window
  • ✓Notes that accounts without sufficient maturation time should not be treated as confirmed negatives
  • ✓Mentions late-arriving labels and proposes a label delay buffer or maturity cutoff
  • ✓Connects false positives/false negatives to review cost and seller experience
2
Training data design and imbalance handling 8-20
  • ✓Explicitly guards against leakage from post-prediction events or investigations
  • ✓Discusses feature consistency between training and serving, especially for early-life activity features
  • ✓Mentions cleaning steps such as deduplication, entity/account linkage, missing values, sparse categories, and outlier handling where relevant
  • ✓Proposes a sensible imbalance strategy such as class weights, stratified/temporal batching, limited resampling, or hard-negative selection
  • ✓Separates probability estimation from decision threshold selection
  • ✓Suggests evaluation metrics appropriate for rare events, such as precision-recall metrics, recall at fixed precision, or alert volume constraints
3
Shift, cold start, and production-readiness trade-offs 20-30
  • ✓Proposes time-based and geography-aware validation slices rather than only random splits
  • ✓Suggests comparing feature/label distributions across countries and time to identify shift
  • ✓Offers a practical cold-start approach for low-label countries, such as using a global model with country features, conservative thresholds, rules plus model, or review-driven label collection
  • ✓Acknowledges uncertainty in sparse-label settings and avoids overclaiming offline confidence
  • ✓Mentions monitoring for drift, calibration, and threshold performance after launch

Run This Interview Live

Start the AI mock interview for this exact scenario. You get unscripted follow-ups, a 30-minute clock, and feedback scored on the same four rubric dimensions. Want to warm up first? Drill the Data Preparation and Class Imbalance question bank, or browse the preparation guides for company-specific formats.

FAQ

Q. What does a Data Scientist class imbalance interview actually test?

It tests data-side judgment before model choice: label definition, leakage control, time-based validation, imbalance handling, and shift in new markets. In this 30-minute blueprint, Interviewer Objectives Alignment and Level-Specific Expectations are 30 points each, so framing and trade-offs outweigh algorithm trivia.

Q. Should I use SMOTE or oversampling for a 0.6% positive rate?

Not as a first move. A strong answer separates probability estimation from threshold selection, tries class weights or limited resampling, and judges the result with precision-recall metrics, not accuracy. Resampling is one option, not the default.

Q. How do you label accounts when fraud confirmation is delayed?

Define an observation window and a future outcome window, then apply a label maturity cutoff. Accounts that have not had enough time to mature should be excluded or held out, never counted as confirmed negatives.

Q. How do you avoid leakage with features from the first 7 days when the business wants earlier predictions?

Fix the prediction time first (signup, day 1, day 3, or day 7), then build training features using only data available at that moment. Features must match what the serving system can compute at inference time.

Q. How do you handle a new country with almost no fraud labels?

Be honest about uncertainty. Practical options are a global model with country features, conservative thresholds, rules plus model, and review-driven label collection, while avoiding strong claims from sparse offline results.

Q. How long is the interview and how is it scored?

The blueprint runs 30 minutes across three phases (0-8, 8-20, 20-30). It is scored out of 100: 30 for objectives alignment, 30 for level-specific expectations, 20 for technical proficiency, and 20 for communication and problem solving.

Before Your Next Screen

Class imbalance questions are rarely about the sampling trick. They are about whether you define the label, the decision point, and the threshold before you touch the model. Practice saying those out loud, in that order, and the follow-ups get easier.

Topics

data scientist interviewclass imbalancedata preparationML interviewmock interviewfraud detectiondistribution shift

Ready to practice?

Put what you've learned into practice with AI mock interviews and structured preparation guides.

Start PreparingBrowse Jobs