Giving and Receiving Feedback Questions

Situation: At my last company I reviewed a pull request for a training pipeline that added a new feature store join and retraining workflow for a customer-churn model. The PR changed data-loading code, added a DAG task, and introduced a custom serialization helper.

Task: My goal was to ensure correctness, reproducibility, and maintainability before merging—especially because this pipeline would run daily and impact production predictions.

Action:

• I ran the branch locally and executed unit tests and a small end-to-end run on a sampled dataset to confirm behavior.
• In the PR comments I gave targeted, constructive feedback:
  • Pointed out a potential race condition in the join logic and suggested a defensive null-check with a small code snippet.
  • Recommended replacing the custom serializer with our existing utils to avoid duplication and improve observability, citing the existing function and linking to docs.
  • Asked for a small integration test that asserts schema invariants after the feature-store join.
  • Suggested a brief design note in the repo describing DAG retry semantics so future engineers understand trade-offs.
• When a teammate argued the custom serializer was slightly faster and clearer, I proposed measuring both approaches on a realistic sample and agreed to prioritize maintainability unless performance differences exceeded 10%. We ran a quick benchmark and found no meaningful difference, so we adopted the shared utility.
• Later, a senior teammate critiqued my suggestion about the retry semantics as overly conservative. I thanked them, re-evaluated failure modes, and updated the design note to a compromise: two retries with exponential backoff and clearer alerting.

Result: The pipeline merged with additional tests and documentation. Production retrains ran for two weeks with zero failures related to the change. The review improved team conventions (we added the serializer utility to the onboarding docs) and reduced future churn.

Learnings: Be specific and evidence-driven in feedback, offer concrete fixes not just problems, and treat disagreements as experiments—measure when possible and be willing to change your stance when shown better data.

Situation: At my previous company I was part of a cross-functional squad building a churn-prediction model. A peer on the team (another data scientist) proposed using a single high-capacity XGBoost model trained on raw customer event counts. I reviewed the code and validation and saw severe target leakage from future events and an unbalanced cross-validation strategy. Pointing this out risked undermining their credibility with the product manager who had already accepted their prototype.

Task: I needed to surface the issues so the model would be reliable in production while preserving our working relationship and the team's confidence.

Action:

• I scheduled a one-on-one, not a public call, to avoid embarrassment.
• I started by acknowledging the strengths of their work (comprehensive feature set, clean code) to show respect.
• I walked through the problematic examples from the codebase with concrete evidence: snippets showing features derived from events after the prediction window and metric differences between naïve and time-series CV.
• I used visual examples (learning curves and leakage examples) to make the issue concrete rather than speculative.
• I proposed concrete alternatives: feature cutoffs, time-based cross-validation, and a simpler baseline model for comparison, and offered to pair-program to fix the pipeline.
• I invited their perspective and adjusted suggestions based on their domain knowledge.

Result: They appreciated the private, evidence-based approach. We refactored the feature pipeline, implemented time-aware CV, and added a baseline model. The revised model had slightly lower apparent training AUC but a 15% improvement in production stability and a 20% reduction in false positives. Our collaboration strengthened—my peer and I later co-authored the modeling guidelines we rolled out squad-wide.

Learning: Preserve trust by separating intent from critique, using data to demonstrate issues, proposing fixes, and collaborating on the solution.

Situation: At my previous company, our team was shipping weekly model updates for churn prediction. During reviews I noticed recurring issues: notebooks with hidden preprocessing steps, no unit tests for feature transforms, and models validated only by accuracy on a single holdout — this caused production incidents where data schema changes silently broke pipelines.

Task: As a senior data scientist, I needed to raise these concerns during a code/model review and drive a process change so we could detect issues earlier and reduce production defects.

Action:

• In a review I documented three concrete problems with examples (undocumented transformations, lack of deterministic preprocessing, and missing tests).
• I proposed and led a short working session demonstrating how a small schema change (added nulls) altered feature distributions and degraded model AUC by 7%.
• Introduced lightweight guardrails: mandatory preprocessing scripts (no hidden cell logic), pytest tests for critical feature transforms, and a standardized model-evaluation checklist including metrics beyond accuracy (AUC, calibration, data drift checks).
• Implemented a pre-merge CI job that runs transform unit tests and validation on a small sample.

Result: Within two sprints adoption was 90% across the team. Production data-related incidents dropped by 60% over the next quarter; mean time to detect issues decreased from days to hours. The team’s review quality improved — reviewers began requesting unit tests and drift checks routinely — and stakeholders reported higher trust in model releases.

This taught me that pairing specific examples with small, automated changes (tests + CI) makes feedback actionable and sustainable.