InterviewStack.io LogoInterviewStack.io
Interview Prep10 min read

SDET Test Coverage Interview: Why 78% Coverage Still Fails

Your suite shows 78% line coverage and checkout bugs still escape. Walk a mid-level SDET test coverage interview turn by turn and see where the 100 points go.

IT
InterviewStack TeamEngineering
|

Dmitri's Plan: Push 78% Coverage Higher While the Double Promo Slips Through

Dmitri walks into a mid-level SDET interview with a tidy answer ready: find the untested lines, write tests, push the percentage up. The checkout team in the scenario already has 78% line coverage, and three bugs still escaped in two months. A prepared candidate is about to lose points on an answer that sounds responsible.

This walkthrough uses the interview package behind InterviewStack.io's AI mock interview for a mid-level SDET on test coverage analysis and optimization. The answers below are illustrative, not a transcript of a real person.

Key Findings

  • The interview runs 30 minutes in 3 phases: 0-8, 8-22, and 22-30 minutes.
  • Level-Specific Expectations and Interviewer Objectives Alignment are worth 30 of 100 points each; Technical Proficiency and Communication are worth 20 each.
  • The scenario gives you 78% line coverage but only 61% branch coverage, a 17-point gap that hides the error paths.
  • The 14-minute strategy phase carries 6 checklist items, the most of any phase.
  • Three incidents in 2 months (double promo, tax rounding, duplicate order) are the evidence you are expected to use.
  • The constraint is fixed: escaped defects must fall with total CI time held flat, against 45 minutes of API tests and a 90-minute nightly end-to-end run.

Interviewer scoring weights across the four rubric dimensions

What Is the Interviewer Asking You to Analyze?

The question sounds like a metrics task but it is a judgment test. Here is the scenario.

The interview question

You are joining a team that owns a checkout service for an e-commerce platform. It handles cart pricing, promotions, tax calculation, payment authorization, and order submission. The team's suite and recent history look like this:

Test suite summary
- Unit tests: ~2,400 tests, 78% line coverage, 61% branch coverage
- API/integration tests: 320 tests, 45 minutes in CI
- End-to-end tests: 70 tests, 90 minutes nightly only
- Recent incidents in the last 2 months:
  1. Promo code applied twice when cart had both subscription and one-time items
  2. Tax rounding bug for a small set of regions
  3. Payment retry path caused duplicate order creation
- Team complaint: CI is slow, but bugs still escape
- Current release goal: reduce escaped defects in checkout without increasing total CI time

How would you analyze the current test coverage and optimize the test strategy for this checkout service to improve confidence before release?

The interviewer is probing whether you read coverage critically, find gaps by risk rather than by percentage, and reason about signal, redundancy, and CI cost. They also want to see how you communicate trade-offs when the budget is fixed.

Four Follow-Ups, Four Slips Where Points Leak

A common first answer is "raise unit coverage toward 90%." The follow-ups are built to expose it.

Turn 1: The 78% Defense

Interviewer: "If the team points to 78% line coverage as evidence the service is already well tested, how would you respond and what additional evidence would you want?"

COMMON MISTAKE
Dmitri says 78% is decent and asks to cover the remaining 22%. That skips branch coverage, error handling, and requirement coverage, so the first-phase checklist item on missing dimensions is lost, along with Interviewer Objectives Alignment points.
STRONGER MOVE
Reframe coverage as a question: which behavior is checked, not which lines ran. Ask for coverage by module, failure history, and a requirement-to-test map, then point at the three incidents as proof that executed code is not verified code.

Turn 2: Choosing the Right Layer

Interviewer: "How would you decide whether a gap should be covered by a unit test, API test, or end-to-end test in this system?"

COMMON MISTAKE
Dmitri says end-to-end is best because that is what users do, and adds more nightly tests. That ignores runtime and flake cost, so the checklist item on moving promo and tax rules to fast layers is missed and Technical Proficiency suffers.
STRONGER MOVE
Place each gap by its failure shape. Deterministic rules like promo stacking and tax rounding belong in unit or service-level tests; keep a small end-to-end set for critical journeys and contract boundaries. Name the trade-off in speed, fidelity, and diagnosability.

Turn 3: Quantifying the Missing Failures

Interviewer: "Suppose you discover many tests exercise the happy path repeatedly but almost none cover failure or retry behavior. How would you quantify and communicate that gap?"

COMMON MISTAKE
Dmitri says the suite needs more negative tests without any measure. With no count of error-path or retry coverage, the answer falls short of the Level-Specific Expectation to communicate why high-risk areas deserve depth despite decent aggregate metrics.
STRONGER MOVE
Build a small matrix of risk area against happy, negative, boundary, retry, and timeout cases, and count what is filled. Present it to product as incident-to-test mapping so the gap is a picture, not an opinion.

Turn 4: Flat CI Budget

Interviewer: "If you are not allowed to increase total CI time, what changes would you make to improve signal while keeping runtime flat or lower?"

COMMON MISTAKE
Dmitri proposes parallelizing and buying faster runners while adding tests on top. That never addresses duplicate happy-path tests across layers, so the pruning checklist item and the runtime-and-CI-cost reasoning behind Interviewer Objectives Alignment are lost.
STRONGER MOVE
Treat minutes as a budget. Find duplicates across layers, consolidate or retire them, push deterministic logic down, and reinvest the saved time in retry and idempotency cases. Roll out module by module, riskiest first.

Why Isn't Reading This Enough?

Spotting these slips on the page is easy. Avoiding them live is the hard part, with a clock running and a follow-up you did not predict. Notice that every slip is a sound instinct delivered without evidence or a trade-off. That habit only changes with reps, which is why the same scenario is worth running out loud.

What Does a Strong SDET Test Coverage Interview Look Like?

This is the blueprint a strong candidate hits, and the exact thing the AI mock interview tracks you against in real time. Phase 1 earns the framing, phase 2 the layer decisions, phase 3 the metrics that cannot be gamed.

The 30-minute interview paced into three phases

Blueprinta strong 30-minute interview, phase by phase
1
Problem framing and coverage assessment 0-8
  • ✓Calls out that 78% line coverage does not guarantee meaningful checkout protection
  • ✓Mentions branch coverage, error handling, state transitions, and requirement coverage as missing dimensions
  • ✓Prioritizes recent incident areas such as promos, tax rounding, payment retry, and duplicate order/idempotency
  • ✓Asks for or states they would inspect failure history, coverage reports by module, and requirement-to-test mapping
  • ✓Recognizes the constraint that CI time cannot increase
2
Optimization strategy and test-layer decisions 8-22
  • ✓Proposes moving deterministic business-rule validation like promo combinations and tax rounding to fast unit or service-level tests
  • ✓Keeps a smaller set of high-value end-to-end tests for critical user journeys and contract boundaries
  • ✓Adds targeted coverage for negative, boundary, retry, timeout, partial failure, and idempotency paths
  • ✓Identifies duplicate happy-path tests across layers and suggests pruning or consolidating them
  • ✓Explains how to cover timing-sensitive payment retry behavior without relying only on brittle end-to-end tests
  • ✓Balances runtime, flake risk, and debugging value when choosing test layers
3
Measurement, trade-offs, and communication 22-30
  • ✓Defines success using outcomes like escaped defect reduction, critical-path regression detection, CI runtime, and flaky failure rate
  • ✓Avoids gaming metrics by explaining why raw coverage increase alone is insufficient
  • ✓Suggests an incremental rollout such as auditing top-risk modules first and replacing low-signal tests gradually
  • ✓Communicates a before-and-after view the team can understand, such as incident-to-test mapping or risk coverage dashboard

Run This Interview Live

Take the same checkout scenario in the AI mock interview. You get unscripted follow-ups, a clock, and feedback scored against these four dimensions. To build your range first, drill the test coverage question bank, then browse SDET prep guides and SDET openings to see what teams ask for.

FAQ

What does an SDET test coverage and optimization interview test?

It tests whether you treat coverage as evidence rather than a score. The 100-point rubric gives 30 points to the interviewer's objectives, 30 to level-specific expectations such as trade-offs and prioritization, 20 to technical proficiency, and 20 to communication and problem solving.

How long is this interview and how is it paced?

It runs 30 minutes in three phases: problem framing and coverage assessment (0-8 minutes), optimization strategy and test-layer decisions (8-22 minutes), and measurement, trade-offs, and communication (22-30 minutes). The middle phase is the longest at 14 minutes and carries 6 checklist items.

Is 78% line coverage good enough to trust a release?

Not on its own. Line coverage says code ran, not that behavior was checked, and branch coverage in this scenario is only 61%. Strong answers ask for coverage by module, failure history, and a requirement-to-test mapping before judging.

How do you decide between a unit, API, or end-to-end test for a gap?

Push deterministic business rules such as promo combinations and tax rounding down to fast unit or service-level tests. Keep a small set of end-to-end tests for critical journeys and contract boundaries, and weigh runtime, flake risk, and debugging value for each gap.

How do you improve quality without increasing CI time?

Reshape the suite instead of growing it: prune duplicate happy-path tests across layers, move logic down to cheaper layers, and spend the freed minutes on retry, timeout, and idempotency paths. Audit the top-risk modules first and replace low-signal tests gradually.

How do you prove a coverage optimization worked?

Track outcomes, not percentages: escaped defects, regression detection on critical paths, CI runtime, and flaky failure rate. A before-and-after incident-to-test mapping gives the team a view they can understand a month later.

Can I practice this exact scenario live?

Yes. InterviewStack.io's AI mock interview runs this checkout coverage scenario, asks unscripted follow-ups based on your answers, and tracks you against the same blueprint in real time.

Where Your Next Rep Goes

Next time someone quotes a coverage percentage, ask what it would have caught. Then say that out loud, under a timer, until it comes out in one breath.

Topics

SDETTest CoverageTest StrategyQA InterviewMock InterviewInterview Prep

Ready to practice?

Put what you've learned into practice with AI mock interviews and structured preparation guides.

Start PreparingBrowse Jobs