Test Case Design and Edge Case Analysis Questions
Systematically deriving the cases, inputs, and conditions most likely to expose defects. Covers formal test-design techniques (equivalence partitioning, boundary value analysis, decision tables, state transitions, and pairwise/combinatorial design) and writing clear, maintainable test cases with documented expected results. Also covers the edge-case mindset: boundary conditions, invalid and unexpected inputs, corner cases, and the attention to detail that anticipates failures when validating complex behavior.
Explain floating-point comparison pitfalls in software, including rounding and representation differences. Provide test strategies and code-level best practices an SDET should apply when writing assertions that compare floats in unit and integration tests, including examples of relative and absolute epsilon checks.
Sample Answer
Direct answer
Floating-point numbers cannot exactly represent most decimal fractions (0.1 in binary floating point is a repeating fraction, just as 1/3 is in decimal), so comparing floats with strict equality (==) is unreliable; the fix is to compare within a tolerance, using a RELATIVE tolerance for large-magnitude values and an ABSOLUTE tolerance as a floor for values near zero, since either one alone fails in a different regime.
Structured elaboration and worked example (executed)
import math
def approx_equal(a, b, rel_tol=1e-9, abs_tol=1e-12):
return math.isclose(a, b, rel_tol=rel_tol, abs_tol=abs_tol)
print("0.1 + 0.2 == 0.3 ->", 0.1 + 0.2 == 0.3)
print("0.1 + 0.2 =", repr(0.1 + 0.2))
print("approx_equal(0.1+0.2, 0.3) ->", approx_equal(0.1+0.2, 0.3))
Running this:
0.1 + 0.2 == 0.3 -> False
0.1 + 0.2 = 0.30000000000000004
approx_equal(0.1+0.2, 0.3) -> True
The strict-equality check is False even though the values are 'the same' for any practical purpose, because 0.1 and 0.2 cannot be represented exactly in binary floating point and their sum accumulates a tiny representation error.
Why relative tolerance alone fails near zero (executed)
a, b = 1e-300, 2e-300
print("math.isclose(a,b) default (rel only) ->", math.isclose(a, b))
print("math.isclose(a,b, abs_tol=1e-12) ->", math.isclose(a, b, abs_tol=1e-12))
Actual output:
math.isclose(a,b) default (rel only) -> False
math.isclose(a,b, abs_tol=1e-12) -> True
Two numbers that are both astronomically small but differ by a factor of 2 (1e-300 vs 2e-300) fail a relative-tolerance-only check, correctly by relative-difference logic, but this is almost always NOT what a test author actually wants: near zero, tiny absolute differences are usually noise, not a meaningful failure, which is why an absolute tolerance floor is needed as a companion check.
Why absolute tolerance alone fails for large numbers (executed)
c, d = 1e15, 1e15 + 100
print("abs diff:", abs(c - d))
print("relative diff:", abs(c - d) / max(abs(c), abs(d)))
print("math.isclose(c, d, rel_tol=1e-9) ->", math.isclose(c, d, rel_tol=1e-9))
print("abs(c-d) < 0.01 ->", abs(c - d) < 0.01)
Actual output:
abs diff: 100.0
relative diff: 9.999999999999e-14
math.isclose(c, d, rel_tol=1e-9) -> True
abs(c-d) < 0.01 -> False
An absolute-tolerance-only check (e.g. abs(a - b) < 0.01) would FAIL this case, flagging two numbers that differ by only about one part in ten trillion as unequal, purely because their magnitude is large; relative tolerance correctly recognizes this as an insignificant difference.
Code-level best practices
- Use a library function (
math.isclosein Python,assertAlmostEqual/an epsilon-based custom matcher elsewhere) rather than hand-rollingabs(a-b) < 0.0001, since a hardcoded absolute epsilon silently breaks at both extremes shown above. - Always pass BOTH
rel_tolandabs_tolexplicitly rather than relying on library defaults, and chooseabs_tolbased on the smallest meaningful magnitude your domain actually produces (a physics simulation and a financial percentage calculation have very different notions of 'negligible'). - Never use exact equality on any float that has passed through at least one arithmetic operation (addition, division, an accumulated sum); exact equality is only safe for a float that was directly assigned a literal and never recomputed.
Trade-offs & pitfalls
A tolerance that is too loose can mask a genuine regression (a calculation that is now systematically off by a small but real amount gets silently accepted), while a tolerance that is too tight reintroduces flaky test failures from ordinary floating-point noise across platforms or numeric library versions; the tolerance value itself is a design decision that belongs in code review, not a default nobody revisits.
For a numeric field 'age' constrained to integers 0 through 120 inclusive, list the set of Boundary Value Analysis test cases you would write. Include the typical BVA set for both single-variable and a small multi-field scenario where 'age' interacts with a 'retirement-flag' enabled for age >= 65. Explain why each BVA case is necessary.
Sample Answer
Direct answer
For an 'age' field valid on integers 0 to 120, the standard BVA set is six values: the invalid value just below the minimum (-1), the minimum itself (0), the value just above the minimum (1), the value just below the maximum (119), the maximum itself (120), and the invalid value just above the maximum (121). When 'age' also drives a second condition (retirement-flag enabled at age >= 65), that second threshold gets its own three-value BVA set around 65.
Structured elaboration
Single-variable BVA for 0-120:
| Test value | Expected result | Why it's needed |
|---|---|---|
| -1 | Rejected | Confirms the lower invalid boundary is actually enforced, not off-by-one (e.g. code that checks age <= 0 instead of age < 0) |
| 0 | Accepted | Confirms the minimum itself is inclusive |
| 1 | Accepted | Confirms the code isn't accidentally excluding the value just inside the range |
| 119 | Accepted | Confirms the code isn't accidentally excluding the value just inside the upper range |
| 120 | Accepted | Confirms the maximum itself is inclusive |
| 121 | Rejected | Confirms the upper invalid boundary is enforced |
Multi-field boundary around the retirement-flag threshold (age >= 65):
| Test value | retirement-flag expected | Why it's needed |
|---|---|---|
| 64 | false | Just below the threshold: flag must stay off |
| 65 | true | The threshold itself is inclusive per the >= operator, so 65 must already be true, not 66 |
| 66 | true | Just above the threshold: confirms the flag stays on, not a one-shot toggle |
Worked example
The reason 65 specifically (not 64 or 66) is the critical case: a common off-by-one bug is writing age > 65 when the requirement says age >= 65, which would silently exclude exactly the 65-year-olds the flag is meant to include. Testing 64 and 66 alone would never reveal that bug, because both would pass under either the correct >= or the buggy > implementation; only testing the exact threshold value distinguishes them. This is why BVA always includes the threshold itself as a test point, not only its neighbors.
Trade-offs & pitfalls
A frequent shortcut is testing only the two invalid boundaries (-1, 121) and skipping the in-range boundaries (0, 1, 119, 120), on the reasoning that "120 obviously works if 119 does." That reasoning is exactly what BVA exists to distrust: inclusive-vs-exclusive comparison bugs live precisely at the boundary itself, not one step away from it. The other common pitfall is treating a derived condition like the retirement flag as covered "for free" by the age boundary tests; it needs its own three-point BVA set at its own threshold (65), because the two thresholds (0/120 and 65) are independent decision points in the code, even though they share the same underlying field.
Given a simple function is_prime(n) in Python, write Pytest unit tests that cover normal cases, edge cases, and performance considerations (e.g., small vs larger inputs). Include at least five tests and explain why each is important.
Sample Answer
Direct answer
A thorough is_prime(n) test suite needs at least: a negative number, the two special small values 0 and 1 (neither prime nor composite by definition), the only even prime (2), an ordinary odd prime, an ordinary composite, and a large input that exercises the algorithm's actual loop bound rather than a value small enough that a broken implementation would pass by coincidence; each of these targets a distinct, plausible off-by-one or definitional bug in a typical trial-division implementation.
Structured elaboration
- Negative numbers: primality is only defined for positive integers greater than 1; a common bug is a modulo-based check that doesn't special-case negatives and returns a wrong answer instead of
False. - 0 and 1: both are explicitly NOT prime by the mathematical definition (a prime must have exactly two distinct positive divisors), but a naive "no divisors found" loop can accidentally return
Truefor both, because a loop likefor i in range(2, n)never executes forn < 2and falls through to a defaultTrue. - 2: the only even prime; a very common bug is an implementation that starts checking divisibility from
2and then also SKIPS even numbers in its loop for performance, incorrectly excluding 2 itself before the loop even gets a chance to confirm it. - A small odd prime and a small composite: baseline correctness on the values the function will overwhelmingly see in normal use.
- A large prime and a large composite with a large prime factor: this is the case that actually tests the algorithm's LOOP BOUND, not just its correctness. A correct trial-division implementation only needs to check divisors up to n, because if n=a×b with both a,b>n, then a×b>n, a contradiction; testing a large composite whose smallest prime factor is itself large (so the loop must run nearly to n before finding it) is what actually exercises that bound, whereas a small composite like
9would pass even with a badly wrong loop bound.
Worked example (executed)
def is_prime(n):
if n < 2:
return False
if n in (2, 3):
return True
if n % 2 == 0:
return False
i = 3
while i * i <= n:
if n % i == 0:
return False
i += 2
return True
Five (plus two extra) pytest tests were written and executed, covering exactly the cases above:
test_negative_number_is_not_prime is_prime(-7) is False
test_zero_and_one_are_not_prime is_prime(0) is False, is_prime(1) is False
test_two_is_prime_and_only_even_prime is_prime(2) is True
test_small_known_primes is_prime(3) is True, is_prime(13) is True
test_small_known_composite is_prime(9) is False, is_prime(100) is False
test_large_prime_performance is_prime(999983) is True
test_large_composite_with_large_prime_factor is_prime(999983 * 2) is False
Executed with pytest: 7 passed. The large-input test uses 999983 (a genuine prime, confirmed by the passing assertion) specifically because its smallest divisor other than 1 is itself, so the loop runs all the way to 999983≈1000 without finding a factor, exercising the loop-bound logic in a way that a small composite like 9 (whose smallest factor 3 is found almost immediately) does not. Per house rule, wall-clock timing was measured out of curiosity during this run but is deliberately NOT reported here, since timing is environment-dependent and unverifiable by a reader re-running the suite; the correctness assertion (is_prime(999983) is True) is what the test actually proves.
Trade-offs and pitfalls
The most common wrong turn is treating "performance considerations" as a wall-clock timing assertion (assert elapsed < 0.01), which is banned as a test design because it is flaky across machines and CI runners; the correct way to test performance-relevant behavior without asserting timing is to test the ALGORITHMIC boundary directly, as the large-prime test does here, by choosing an input specifically engineered to force the loop to its true upper bound rather than exit early. A second pitfall is conflating "large" with "slow": for trial division, a large NUMBER that happens to have a small factor (e.g. 999983 * 2, which is even) is fast to reject, while a large PRIME is the actually expensive case, so test data must be chosen by factorization structure, not just magnitude.
Implement a Python function robust_mean(values: List[Optional[float]]) -> float that computes the arithmetic mean while ignoring None and NaN. It should raise ValueError if no valid numbers are present. Consider numerically stable accumulation to reduce overflow risk when values have large magnitude; mention algorithm choice (Kahan or incremental). State complexity, behavior on infinities, and assumptions about IEEE floats.
Sample Answer
Direct answer
robust_mean should use Kahan summation to control floating-point accumulation error on large or high-magnitude inputs, skip None and NaN values while counting only the valid ones, raise ValueError when no valid numbers remain, and treat infinities as a special case rather than letting them flow through Kahan's compensation arithmetic, which silently produces NaN if left unguarded.
Structured elaboration and worked example (executed)
import math
from typing import List, Optional
def robust_mean(values: List[Optional[float]]) -> float:
total = 0.0
compensation = 0.0
count = 0
saw_pos_inf = False
saw_neg_inf = False
for v in values:
if v is None:
continue
if isinstance(v, float) and math.isnan(v):
continue
if v == float('inf'):
saw_pos_inf = True; count += 1; continue
if v == float('-inf'):
saw_neg_inf = True; count += 1; continue
y = v - compensation
t = total + y
compensation = (t - total) - y
total = t
count += 1
if count == 0:
raise ValueError("no valid numbers present")
if saw_pos_inf and saw_neg_inf:
return float('nan')
if saw_pos_inf:
return float('inf')
if saw_neg_inf:
return float('-inf')
return total / count
The infinity bug this design deliberately avoids
A first implementation without the explicit infinity branches was tried and FAILED at execution: calling robust_mean([float('inf'), 1, 2]) returned nan instead of inf. The root cause is that Kahan's compensation step computes (t - total) - y; the moment an infinite value flows through it, t becomes inf, and inf - inf (inside that compensation formula) evaluates to nan, which then poisons every subsequent element because compensation stays nan for the rest of the loop. The fix shown above tracks infinities OUTSIDE the Kahan accumulator entirely and reasons about them algebraically afterward (both signs present -> nan, since +inf and -inf together have no defined mean; one sign present -> that infinity dominates the mean regardless of the finite values).
Demonstrated numerical stability gain (executed, not asserted)
N = 10_000_000
vals = [0.1] * N
naive_total = 0.0
for v in vals:
naive_total += v
naive_mean = naive_total / N
kahan_mean = robust_mean(vals)
print("naive_running_mean =", repr(naive_mean))
print("naive abs error from 0.1 =", abs(naive_mean - 0.1))
print("kahan robust_mean =", repr(kahan_mean))
print("kahan abs error from 0.1 =", abs(kahan_mean - 0.1))
Actual output:
naive_running_mean = 0.09999999998389754
naive abs error from 0.1 = 1.6102466582346153e-11
kahan robust_mean = 0.1
kahan abs error from 0.1 = 0.0
Summing ten million copies of 0.1 with a naive running total gives naive_running_mean = 0.09999999998389754, an absolute error of about 1.6e-11 from the true value 0.1. The Kahan-based robust_mean on the identical input returned exactly 0.1, a measured error of 0.0. This is not a marginal difference: naive summation accumulates rounding error roughly proportional to the number of additions, while Kahan's compensation term cancels the dominant rounding error at each step, which is why it is the standard choice once an aggregation runs over many elements or elements of very different magnitudes.
Complexity and assumptions
O(n) time and O(1) additional space, a single pass over the input. The function assumes IEEE-754 double-precision floats (Python's native float), so float('nan') != float('nan') and NaN-detection must use math.isnan, never ==; it also assumes the caller wants NaN and None values silently excluded rather than treated as an error, which is a documented contract choice, not a universal default (a stricter variant might instead raise on the first NaN it encounters).
Trade-offs & pitfalls
Without the explicit infinity handling, this exact function would ship a plausible-looking, executes-without-error implementation that returns a silently wrong nan for a perfectly reasonable input (one infinite value among otherwise-finite ones) instead of raising or returning inf; this is precisely the kind of bug that a code-reading review would very likely miss, because Kahan summation's compensation formula reads as obviously correct arithmetic unless you specifically trace what happens when t becomes infinite.
Given a function reverse_string(s) in Python that returns s reversed, list and explain all edge and corner cases you would test for this simple API. Then write three concrete pytest unit tests (inputs and expected outputs) that specifically cover tricky cases such as empty string, combining/unicode characters (emojis, surrogate pairs), and very long inputs. Focus on the test cases and rationale rather than implementing the function itself.
Sample Answer
Direct answer
A naive reverse_string(s) = s[::-1] correctly reverses ASCII text, but breaks on inputs whose visible characters are made of MULTIPLE unicode codepoints (combining marks, and some emoji), because slicing reverses codepoints individually rather than the visual units a person reads; the essential edge cases are therefore the empty string, single-codepoint text, multi-codepoint 'characters' (combining marks and certain emoji), and very long inputs for performance.
Structured elaboration and worked example (executed, bug confirmed)
def reverse_string(s):
return s[::-1]
def test_empty_string():
assert reverse_string("") == ""
def test_simple_ascii():
assert reverse_string("hello") == "olleh"
def test_single_emoji_codepoint():
assert reverse_string("a\U0001F600b") == "b\U0001F600a" # single-codepoint emoji reverses fine
def test_flag_emoji_multi_codepoint():
us_flag = "\U0001F1FA\U0001F1F8" # Regional Indicator U + S = US flag
reversed_flag = reverse_string(us_flag)
swapped = "\U0001F1F8\U0001F1FA" # Regional Indicator S + U = a different regional-indicator pair
assert reversed_flag == swapped
assert reversed_flag != us_flag # BUG: reversing changes the flag's MEANING
Running this: all four assertions pass, INCLUDING the assertion that confirms the bug (reversed_flag != us_flag). Printed codepoints:
original codepoints: ['0x1f1fa', '0x1f1f8']
reversed codepoints: ['0x1f1f8', '0x1f1fa']
Why this matters
A flag emoji like the US flag is not one character to the underlying string representation, it is two 'Regional Indicator Symbol' codepoints (U+1F1FA and U+1F1F8) that a rendering engine combines into a single flag glyph. Reversing the CODEPOINT sequence with s[::-1] swaps their order, and because those two codepoints combine into a DIFFERENT valid flag when swapped, the function doesn't just produce visual garbage, it silently changes the string's MEANING to a different, equally-valid-looking flag. This is worse than a crash: it is a silent semantic corruption that a purely visual QA pass could easily miss.
Three pytest cases covering tricky inputs
def test_reverse_empty():
assert reverse_string("") == ""
def test_reverse_flag_emoji_documents_bug():
us_flag = "\U0001F1FA\U0001F1F8"
# This assertion documents CURRENT (buggy) behavior; a fixed implementation
# using grapheme-cluster-aware reversal should instead assert equality to us_flag.
assert reverse_string(us_flag) != us_flag
def test_reverse_very_long_input():
long_s = "ab" * 500_000
result = reverse_string(long_s)
assert result[:2] == "ba" and len(result) == len(long_s)
Trade-offs & pitfalls
The fix for grapheme-cluster correctness (using a unicode-segmentation library to reverse by user-perceived character rather than by codepoint) is a real engineering trade-off: it adds a dependency and CPU cost that most string-reversal use cases (e.g. reversing an internal identifier) do not need, so the right answer depends entirely on whether the function is ever applied to human-authored text containing emoji or combining marks; a function documented as 'reverses ASCII identifiers only' does not need this fix, but one exposed as a general-purpose text utility does, and the edge-case list above is exactly what surfaces that scoping question before it becomes a production surprise.
Unlock Full Question Bank
Get access to all Test Case Design and Edge Case Analysis interview questions and detailed answers.
Sign in to ContinueJoin thousands of developers preparing for their dream job.