Mocking, Stubbing, and Test Isolation Questions

Techniques for isolating tests from external dependencies using mocks, stubs, and test doubles. Understanding when to mock vs. when to use real services, and how to make tests reliable while still validating real behavior.

MediumTechnical

50 practiced

Your team relies heavily on mocks and a production bug escaped because an over-specified mock hid integration behavior. As SDET propose a practical 3-month plan to rebalance testing (add integration tests, contract tests, golden-path e2e), include measurable KPIs to track improvement and specific migration steps to convince stakeholders.

Sample Answer

**Situation & goal (1 sentence)** As SDET I’ll rebalance from over-mocking to a layered strategy (contract + integration + golden-path E2E) so we catch integration regressions earlier and keep fast feedback.

**3-month practical plan (high-level weeks)** - Month 1 — Discover & quick wins - Inventory existing mocks, identify top 10 high-risk mocks tied to production incidents. - Add contract tests (PACT or similar) for services behind those mocks. - KPI: 10 contracts in CI, contract test pass rate 100%. - Month 2 — Integration coverage & automation - Implement lightweight integration tests (db + service) for 8–12 critical flows; run in nightly pipeline. - Introduce consumer-driven contract verification into PR gates for new changes. - KPI: Nightly integration coverage for 80% of critical flows; mean time to detect (MTTD) for integration bugs decreases by 30%. - Month 3 — Golden-path E2E & stabilization - Build 15 golden-path E2E scenarios that run in pre-prod on deploy and smoke on PRs where feasible. - Measure flakiness and reduce false positives via test hardening. - KPI: E2E flake rate <5%; production escapes from tested areas = 0 for 30 days.

**Migration steps to convince stakeholders** - Present incident post-mortem showing how over-specification hid behavior; map to business impact (customer incidents, hours lost). - Pilot: propose one-service pilot (2 weeks) to show contract + integration reduces regressions — share metrics. - Cost/benefit: estimate CI cost vs. reduced rollback/incident cost. - Governance: require contract tests for any API change; add visibility dashboard.

**Risks & trade-offs** - Slower CI for full E2E: mitigate by targeted golden-paths and gating. - Initial effort: align with product, schedule pilot on non-peak sprint.

**Expected measurable outcomes** - MTTD down 30–50%, production bug escapes down 80% in tested areas, E2E flake <5%, contract coverage for 100% of critical integrations.

EasyTechnical

57 practiced

Describe test isolation in the context of automated testing for a microservice. Explain why isolation matters, list common sources of test interference (shared state, network, time, randomness), and outline SDET practices to enforce isolation in CI pipelines to reduce flakiness and false positives.

EasyTechnical

60 practiced

Define test flakiness and describe three ways mocks can both reduce and introduce flakiness. For each item provide an SDET mitigation recommendation to keep the test suite stable and predictable over time.

HardTechnical

43 practiced

Design an automated process that compares mock expectations used in tests with production behavior by consuming telemetry, API logs, or sampled production traffic. Describe data collection, privacy-preserving sampling, comparison algorithms for field-level diffs, thresholds for CI alerts, and ways to minimize false positives.

Sample Answer

**Situation / Goal (brief)** As an SDET I’d build an automated pipeline that continuously validates test mocks against production behavior using telemetry, API logs, and sampled traffic to catch drift early without exposing PII.

**Data collection** - Ingest structured API logs, request/response telemetry, and sampling of payloads into a staging data store (Kafka → Parquet on S3 or BigQuery). - Enforce schema normalization (field names, types, timestamps) at ingestion.

**Privacy-preserving sampling** - Apply client-side or edge sampling: reservoir sampling + rate limits per user to avoid bias. - Redact or hash PII fields (e.g., email, ssn) using deterministic hashing + tokenization and store only schema/type metadata for sensitive fields. - Only capture example shape for array/binary blobs; enforce retention windows and access controls.

**Comparison algorithm (field-level diffs)** - For each mock endpoint, group production samples by API version and route. - Normalize types and run field-level comparison: existence diff, type diff, enum/value-set diff, schema evolution (optional fields). - Use fuzzy comparison for numeric tolerances and timestamp skews; for nested objects use recursive diff producing path-based deltas. Example metric: fraction_of_samples_with_field_change.

**Thresholds for CI alerts** - Define multi-tier thresholds: - Blocker (fail CI): breaking changes (required field removed, type changed) affecting >1% of samples. - Warning (post-merge alert): behavioural diffs (new optional fields, new enum values) in 1–5% of samples. - Info: rare deviations <1%. - Allow per-endpoint overrides and rollout windows.

**Minimizing false positives** - Use historical baseline and smoothing (7–14 day rolling window) to avoid transient blips. - Correlate diffs with deployments/feature flags; suppress alerts during known deploy windows. - Apply confidence scoring: require consistent change across multiple samples and clients before raising high-severity alerts. - Provide rich context in reports: sample variance, example payloads (redacted), and suggested mock updates.

**Implementation notes** - Integrate as a CI check that queries the latest production sample snapshot, emits SARIF-compatible report; failing builds link to a dashboard for triage. - Start with critical endpoints, iterate thresholds with stakeholders.

HardTechnical

53 practiced

Design tests that simulate partial failures across distributed transactions (for example implementing saga patterns) by using mocks or service virtualization. Ensure tests validate idempotency, compensating actions, and recovery on process restart. Provide concrete test scenarios, orchestration approach, and how to automate them safely in CI.

Unlock Full Question Bank

Get access to hundreds of Mocking, Stubbing, and Test Isolation interview questions and detailed answers.

Join thousands of developers preparing for their dream job.