Code Quality & Technical Communication Questions

Best practices and principles for writing clean, maintainable code and communicating technical decisions clearly. Topics include code quality metrics, code reviews, refactoring, static analysis, testing strategies related to maintainability, documentation standards, API/documentation practices, and effective communication of design and architecture decisions.

HardTechnical

49 practiced

How would you set realistic test coverage targets for a heterogeneous data science repository that contains notebooks, ETL scripts, feature engineering pipelines, model training code, and production scoring modules? Propose a measurement approach (what to include/exclude), a coverage policy for PRs, and alternatives to line coverage that better reflect quality (e.g., mutation testing, critical-path tests).

Sample Answer

Measurement approach — what to include/exclude- Include: production ETL scripts, feature-engineering pipelines, model training code (unit+integration), production scoring modules, reusable utility libraries. These directly affect correctness, performance, and data contracts.- Exclude or treat differently: exploratory notebooks (convert important ones to tests if they encode logic), ad-hoc analysis, long-running backfill jobs (measure via integration tests or end-to-end validation rather than line coverage).- Categorize files by risk: Critical (scoring/ETL/data contracts), Medium (training/preprocessing), Low (visualizations, experiments). Apply different target bands.

Coverage targets (realistic, risk-based)- Critical: 85–95% statement/branch coverage; mandatory CI gates.- Medium: 70–85% coverage; require tests for core logic and data contracts.- Low: 40–60% or no hard target; encourage tests for reusable pieces.- Repo-wide aspirational target: ~75% but not enforced globally—enforce per-category.

PR policy- Every PR must include tests for new/changed functionality; CI fails if changes touch Critical files without test additions.- Require unit tests for pure functions, integration tests for data pipelines, smoke tests for scoring endpoints.- Use review checklist: data contract changes, model checkpoint changes, schema migrations must include regression tests or dataset fixtures.- Allow deliberate exceptions via documented risk assessment and approver sign-off.

Alternatives / complements to line coverage- Mutation testing to evaluate test effectiveness for core modules (run on Critical and Medium).- Contract/schema tests: assert input/output shapes, types, nullability, ranges.- Critical-path tests: end-to-end DAG runs on sampled production-like data (fast subset) to catch pipeline regressions.- Property-based tests for preprocessing/feature transforms.- Quality gates: model metric regression checks, data drift detectors, and synthetic adversarial tests.- Test prioritization: focus CI on fast unit suites, nightly mutation and E2E runs, and staged rollout with canary scoring.

Instrumentation & dashboards- Track per-file/category coverage, mutation score, test run times, and flaky-test rate. Use these to adjust targets and focus testing investment.- Periodically review targets with stakeholders; evolve based on incident history and team capacity.

This approach balances risk, developer velocity, and meaningful test quality beyond raw line coverage.

MediumTechnical

60 practiced

You inherit a legacy model training codebase with very high cyclomatic complexity and low test coverage. Propose a step-by-step, low-risk refactoring plan that increases test coverage and reduces complexity while minimizing impact on production. Include concrete steps like characterization tests, refactoring techniques, and deployment safeguards.

Sample Answer

1) Clarify constraints & goals- Confirm production SLAs, retraining cadence, acceptable model-performance delta, and rollback policy with stakeholders.

2) Characterization tests (safety net)- Add end-to-end regression tests that assert model outputs on canonical datasets (store inputs, expected predictions, metrics).- Add data-contract tests (schema, distributions, null rates) using pytest and Great Expectations.- Capture current model evaluation metrics (AUC, RMSE, feature importances) as baselines.

3) Incremental refactoring plan (low-risk, small PRs)- Identify modules with highest cyclomatic complexity (radon/flake8 complexity report).- Apply "extract function" to isolate pure functions (data transforms, loss computation) and write unit tests for each extracted piece.- Replace long conditionals with polymorphism or strategy pattern (simple classes/functions) one conditional at a time.- Modularize I/O: wrap data loading and feature pipelines behind interfaces so they can be mocked.- Move side-effecting code (file writes, logging, training loops) into thin outer layers.

4) Test-first for risky areas- For each refactor, write characterization tests that assert behavior before change.- Run tests locally and in CI; require 100% pass for merge.

5) Continuous integration and deployment safeguards- Add CI pipeline stages: lint -> unit tests -> integration/regression tests -> model-eval.- Create a shadow rollout: deploy refactored model in parallel (A/B or canary) logging inputs/outputs, compare drift and performance.- Implement automated rollback if performance below threshold or data-contract violations occur.

6) Monitoring & gradual cleanup- After stable canary, increment traffic and remove legacy paths.- Track code complexity metrics and test coverage per PR; set targets (e.g., <10 cyclomatic per function, >80% coverage).- Document design, data contracts, and runbook for incidents.

Why this works:- Characterization tests preserve behavior; small, tested refactors minimize risk; modularization improves testability; shadow/canary reduces production impact while enabling verification.

HardTechnical

56 practiced

A feature engineering pipeline implemented as chained pandas operations fails with OOM errors on large datasets. Propose a refactor plan and provide example code sketches to move to a scalable approach (chunking with pandas, Dask, or pushing transformations to SQL). Explain correctness checks and performance validation steps.

Sample Answer

Approach summary:- Replace an in-memory chain of pandas operations with a pipeline that scales: (1) push transformations into SQL where possible, (2) chunked pandas processing when SQL not feasible, (3) use Dask for larger-than-memory DataFrame-style workflows. Validate correctness at each stage and benchmark.

Example 1 — Push to SQL (preferred for aggregations/filter/join):

sql

-- create a materialized view with heavy joins/aggregates
CREATE MATERIALIZED VIEW fe_base AS
SELECT t.user_id,
       COUNT(*) FILTER (WHERE event='purchase') AS purchases,
       AVG(value) AS avg_value,
       MAX(ts) AS last_ts
FROM events t
WHERE ts >= '2023-01-01'
GROUP BY t.user_id;

Fetch small result set for ML.

Example 2 — Chunked pandas (when custom py logic required):

python

import pandas as pd

def process_chunk(chunk):
    # custom transformations, avoid .apply on rows when vectorized ops available
    chunk['log_val'] = np.log1p(chunk['value'])
    return chunk.groupby('user_id').agg({'log_val':'mean'})

reader = pd.read_csv('events.csv', chunksize=2_000_000)
chunks = []
for ch in reader:
    chunks.append(process_chunk(ch))
fe = pd.concat(chunks).groupby('user_id').mean()  # final reduce

Example 3 — Dask (when pipeline structure mirrors pandas but data larger than RAM):

python

import dask.dataframe as dd
df = dd.read_parquet('s3://bucket/events/')
df = df[df.ts >= '2023-01-01']
df['log_val'] = df['value'].map_partitions(np.log1p)
fe = df.groupby('user_id').log_val.mean().reset_index().persist()
fe.to_parquet('s3://bucket/fe_user/')

Correctness checks:- Unit-test transformations on small sampled dataset; compare pandas result vs SQL/Dask/chunked pipeline using assert_frame_equal (tolerances for floats).- Row-count conservation checks, key-uniqueness, summary-statistics (min/max/mean) before/after.- Use data diff (left-anti join) to find mismatches.

Performance validation:- Measure peak memory (psutil) and wall time for sample runs.- Benchmark scalability: run on 1x, 5x, 10x data; plot time/mem vs size.- For Dask: tune partitions (bytes/partition ~100MB), use persist(), profile with Dask dashboard.- For SQL: compare execution plans, add indexes/materialized views; measure query latency and I/O.Trade-offs: SQL reduces Python overhead but limited for complex custom functions; chunking simpler but I/O heavy; Dask easier API-migration but requires cluster/tuning.

HardSystem Design

58 practiced

Design the structure and CI/CD pipeline for a multi-model serving platform that must support: canary releases for models, A/B experiments, multi-region deployment, rollback, and experiment-specific feature flags. Describe repo structure (monorepo vs polyrepo), deployment artifacts, model registry interactions, and automated checks that must pass before canary rollout.

Sample Answer

Requirements (brief):- Canary releases, A/B experiments, multi-region, rollback, experiment-specific feature flags, reproducibility, auditability.

Repo strategy:- Hybrid: polyrepo for model dev (each data science team owns a model repo with training code, tests, and CI). Monorepo for platform/infra (deployment charts, operator code, admission policies, shared SDKs, feature-flag integration). Reasons: DS teams iterate fast; platform changes need coordinated CI and strong governance.

Deployment artifacts:- Model package: versioned artifact in model registry (serialized model + metadata: schema, signature, training data snapshot, metrics, provenance).- Container image: lightweight serving image (base runtime) + model artifact injected at deploy time or referenced by immutable URI.- Infra manifests: Helm charts/Kustomize and Terraform modules per-region; experiment config (routing, traffic splits, feature-flag keys).- Canary config: rollout policy (percent steps, duration), health metrics, SLOs.

Model registry interactions:- Registry stores artifacts, metadata, lineage, and SLO/perf baseline.- CI promotes artifact: validation → signed artifact → "staged" in registry → gated release to canary.- Registry exposes immutable URIs and exposes model metadata via API to runtime for explainability/audit.

CI/CD pipeline (high-level):1. Model dev CI (per-model repo) - Unit tests, static checks, reproducible training (seeded), packaging. - Compute model card: dataset version, features, hyperparams, training metrics. - Push artifact to model registry (with hash + signature).2. Validation pipeline (central) - Automated checks (must pass before canary): - Schema / contract checks (input/output types) - Offline performance tests vs baseline (AUC, RMSE) on holdout and production shadow data - Data drift / feature distribution checks between training and recent production data - Bias and fairness scans required by policy - Explainability sanity (SHAP/feature importance smoke tests) - Resource/cold-start performance benchmark (latency under p95) - Security scan of container and dependency SBOM - Unit / integration tests for SDK and API compatibility - Model card and approvals (manual sign-off for high-risk models) - If all pass -> promote to "canary-ready".3. Canary rollout (automated) - Deploy model to canary instances (per region). Use feature-flag/traffic manager to route X% traffic to canary for specific experiment IDs or A/B cohorts. - Shadow mode option: mirror traffic without affecting responses. - Monitor real-time metrics: latency, error-rate, business KPIs, calibration, feature-level distributions. - Automated gating: proceed to next percent only if health and KPI thresholds hold for configured window; otherwise auto-rollback.4. Gradual promotion to prod across regions or A/B cohorts, update model registry status.

Multi-region considerations:- Per-region clusters with regional model caches; global config via control plane (GitOps + ArgoCD/Gloo/Envoy).- Replicate registry metadata and artifacts across regions with consistent hashing or CDN.- Local canary in each region with same gating rules to respect regional data/regulations.

A/B experiments & feature flags:- Use experiment platform or FF service: route cohorts using experiment IDs; model selection and flag keys tied to registry artifact id and experiment config.- Store experiment config in monorepo or central config service; CI updates experiment definitions via PR reviewed pipeline.

Rollback and audit:- Rollback triggered by automated gates or manual abort; system re-routes to previous artifact URI and scales down canary.- All deployments logged in registry + audit trail (who, when, metrics).- Retain previous artifacts and environment snapshots for quick redeploy.

Observability & SLOs:- Metrics: model-specific metrics (prediction distribution), business metrics, infra metrics.- Alerting based on composite signals; automated diagnostics (drift detector, explainability diffs) on anomalies.

Why this design:- Hybrid repo gives DS velocity and central governance.- Model registry + immutable URIs guarantee reproducibility and safe rollbacks.- Multi-stage CI ensures models meet statistical, fairness, performance, and security requirements before any user traffic.- Canary + feature flags enable controlled experiments and safe rollouts per-region and per-cohort.

HardTechnical

53 practiced

You must remove a widely used in-place mutating utility function that is unsafe and causes subtle bugs across many downstream projects. Draft a complete deprecation and removal plan that minimizes disruption: steps for deprecation warnings, migration helpers, versioning strategy, tests to validate consumer migrations, rollout timeline, and communication plan to dependent teams.

Sample Answer

Situation: We're removing a widely used in-place mutating utility (e.g., df.fillna_inplace, mutate_array) that causes subtle bugs in pipelines and model training due to unexpected state changes.

Goal: Minimize disruption across data science codebases, maintain reproducibility, and provide a clear migration path.

Plan (high-level):1. Deprecation phase (3 months)- Release minor version X.Y where the function remains but emits a clear DeprecationWarning at call time, including: - Name of replacement (non-mutating variant) - Link to migration guide and codemod - Example stacktrace hint- Add runtime flag/environment variable DISABLE_MUTATING_UTILS=false to let teams opt-in to strict checks.- Add CI linter rule (flake8/pylint plugin) that flags usage.

2. Migration helpers- Provide replacement API: a pure function that returns new object (e.g., fillna(df) -> new_df).- Provide convenience wrapper that asserts equality of behavior for a short period: - migrate_fillna(df, inplace=False) with warnings.- Publish codemod scripts (python script using lib2to3 or rope) to: - Replace df.mutate_inplace(...) → df = mutate(df, ...) - Update variable usage when chained calls are present.- Provide checklist and examples for common patterns (notebooks, pipeline steps, feature stores).

3. Versioning & release strategy- Deprecation release: minor bump X.Y (backward compatible).- Removal release: next major bump X+1.0 after 3 months.- Provide a beta/opt-in preview in a pre-release (X.Y+0.1rc) for teams to test.

4. Tests to validate consumer migrations- Provide a test suite template consumers can drop into their repo: - Behavioral tests comparing old vs new outputs on representative datasets (small fixtures). - State/identity tests ensuring original objects are unchanged by new API. - Reproducibility tests for pipelines and model training (seeded RNG) to detect unintended side-effects.- Provide example pytest fixtures and CI config (GitHub Actions snippet) to run these tests automatically.- Offer a migration validation script that scans codebase for patterns and runs inference on sample pipeline to assert identical outputs within tolerances.

5. Rollout timeline (example)- Week 0: Announce plan to stakeholders, publish migration guide and codemod.- Weeks 1–12 (Deprecation window): - Week 1: Release X.Y with warnings + linter plugin. - Weeks 2–6: Host office hours, internal webinars, Slack channel; collect blocker reports. - Week 6: Publish codemod v1 and sample migrations for top 10 repos. - Weeks 7–12: Encourage teams to adopt; run migration tests; provide hands-on help for critical pipelines.- Week 13: Cut release candidate for removal and publish migration report.- Week 14: Release X+1.0 that removes the function.- Post-release (4 weeks): Support period for hotfixes; revert plan if major breakages discovered.

6. Communication plan- Stakeholder map: data science teams, ML infra, feature store owners, production engineers, data platform.- Channels & cadence: - Kickoff email & doc (Week 0) with migration guide, codemod, timeline. - Weekly status updates on project board and dedicated Slack channel. - Calendar invites for migration office hours and 1:1 sessions for high-risk teams. - Release notes in package repo linking to tests and examples.- Escalation: designate owners (one engineering lead, one senior DS) for triage; weekly sync with product/engineering managers.- Metrics to track adoption: number of repos migrated, lint warnings remaining, CI failures pre/post migration, number of support tickets.

Why this minimizes disruption:- Gives teams time with automated tooling (codemod) and tests to validate behavior.- Non-mutating API preserves reproducibility and makes side effects explicit.- Linter and CI tests create automated safety nets.- Communication and hands-on support reduce risk to critical pipelines and models.

Example migration snippet:

python

# old (mutating)
df.fillna_inplace(0)

# new (non-mutating)
df = df.fillna(0)

This plan balances a clear deadline (major release removal) with tooling, tests, and support to ensure smooth transition while preserving model correctness and reproducibility.

Unlock Full Question Bank

Get access to hundreds of Code Quality & Technical Communication interview questions and detailed answers.

Join thousands of developers preparing for their dream job.