Edge Case Identification and Testing Questions

Focuses on systematically finding, reasoning about, and testing edge and corner cases to ensure the correctness and robustness of algorithms and code. Candidates should demonstrate how they clarify ambiguous requirements, enumerate problematic inputs such as empty or null values, single element and duplicate scenarios, negative and out of range values, off by one and boundary conditions, integer overflow and underflow, and very large inputs and scaling limits. Emphasize test driven thinking by mentally testing examples while coding, writing two to three concrete test cases before or after implementation, and creating unit and integration tests that exercise boundary conditions. Cover advanced test approaches when relevant such as property based testing and fuzz testing, techniques for reproducing and debugging edge case failures, and how optimizations or algorithmic changes preserve correctness. Interviewers look for a structured method to enumerate cases, prioritize based on likelihood and severity, and clearly communicate assumptions and test coverage.

HardTechnical

73 practiced

Explain numeric edge cases introduced by mixed-precision training (fp16) and quantization (INT8). Design tests that detect gradient underflow, overflow, sudden loss spikes, and severe accuracy degradation after quantization. Suggest mitigations such as dynamic loss scaling, clipping, and calibration, and describe how you would validate each mitigation with tests.

Sample Answer

Mixed-precision (fp16) and post-training quantization (INT8) introduce numeric edge cases because reduced mantissa/exponent range decreases dynamic range and precision. Common issues:- Gradient underflow: small gradients round to zero in fp16 (especially for deep nets / small LR).- Overflow/Inf/NaN: large activations or gradients exceed fp16 max → Inf, breaking optimizer.- Sudden loss spikes: accumulation of rounding error or non-deterministic reductions cause training instability.- Severe accuracy degradation after INT8 quantization: weight/activation rounding and distribution shift reduces representational fidelity.

Tests to detect problems1. Gradient underflow- Instrument training to log per-layer gradient magnitudes and count zeros. Run short training with fp32 baseline and fp16. Flag if >X% more zero gradients in fp16 or if step-wise weight updates fall below tolerance.2. Overflow/Inf detection- During forward/backward, assert no Inf/NaN in activations, gradients, optimizer states. Use synthetic inputs scaled to stress ranges; expect no NaNs.3. Sudden loss spikes- Run multiple seeds and track loss trajectory; compute moving-window variance. Flag spikes where loss increases >Y*std or >Z% relative to previous epoch.4. Post-quantization accuracy degradation- Compare validation metrics of fp32 vs INT8 on held-out set; compute per-class accuracy drop and KL divergence between softmax outputs. Flag if overall accuracy drop > threshold.

Mitigations and validation- Dynamic loss scaling (fp16): multiply loss by scale S, backprop, then unscale gradients; adapt S up/down on success/overflow. Validate by repeating overflow test: dynamic scaling should eliminate NaNs and restore comparable gradient non-zero rates to fp32.- Clipping (grad and activation): apply gradient norm clipping and activation clipping (for known layers like GELU). Validate by rerunning loss-spike and overflow tests; clipping should reduce spikes and prevent overflows while preserving convergence speed within tolerance.- Mixed-precision with master fp32 weights: keep optimizer states and weight copies in fp32 while computing in fp16. Validate by comparing final accuracy and by ensuring gradients/updates are numerically close to full fp32 baseline (L2 diff below threshold).- INT8 calibration (per-channel, KL or MSE): collect activation histograms on calibration set, choose scales per-channel, optionally use bias correction or quant-aware training (QAT). Tests: post-calibration inference should reduce KL divergence and accuracy drop; QAT should converge back to within target delta of fp32. Also run adversarial/edge-case inputs to ensure no catastrophic outputs.- Automated regression suite: integrate the above tests into CI with synthetic stress tests, small real-data runs, and deterministic seeds. Include thresholds (accuracy delta, NaN counts, gradient sparsity) and require mitigation passes before merge.

Key reasoning: detect instrumentally (metrics, histograms, assertions), reproduce with stress inputs, and validate mitigations by rerunning the exact failing tests and measuring closeness to fp32 baseline.

EasyTechnical

82 practiced

Write two pytest tests (pseudo-code acceptable) that verify a saved model can be serialized and deserialized without significantly changing deterministic predictions. Use scikit-learn or PyTorch semantics and include: (1) save/load preserving outputs within tolerance for numeric models, and (2) behavior when the saved file is missing or corrupted. State expected tolerances and behaviors.

Sample Answer

To verify serialization preserves deterministic predictions and handles missing/corrupted files, two focused pytest tests are sufficient.

`python

# python
import numpy as np
import joblib
import pytest
from sklearn.linear_model import LogisticRegression
from sklearn.datasets import make_classification
from pathlib import Path

def test_save_load_preserves_predictions(tmp_path):
    # Arrange: train a deterministic sklearn model
    X, y = make_classification(n_samples=200, n_features=10, random_state=42)
    model = LogisticRegression(random_state=0, solver='lbfgs', max_iter=1000)
    model.fit(X, y)
    inputs = X[:20]
    baseline_preds = model.predict_proba(inputs)  # numeric outputs

    # Act: save and load
    p = tmp_path / "model.joblib"
    joblib.dump(model, p)
    loaded = joblib.load(p)
    loaded_preds = loaded.predict_proba(inputs)

    # Assert: predictions preserved within tolerance
    # Expected tolerances: relative tol 1e-6, absolute tol 1e-8
    assert np.allclose(baseline_preds, loaded_preds, rtol=1e-6, atol=1e-8)

def test_load_missing_or_corrupted_file_behaviour(tmp_path):
    # Missing file -> should raise FileNotFoundError when loading
    missing = tmp_path / "no_such_model.joblib"
    with pytest.raises(FileNotFoundError):
        joblib.load(missing)

    # Corrupted file -> loading should raise an error (ValueError / EOFError / RuntimeError)
    corrupted = tmp_path / "corrupt.joblib"
    corrupted.write_bytes(b"not-a-valid-joblib")  # create corrupted file
    with pytest.raises((ValueError, EOFError, RuntimeError, OSError)):
        joblib.load(corrupted)

Key points:- Use predict_proba for numeric stability check; for deterministic class labels you can assert exact equality.- Tolerances: rtol=1e-6, atol=1e-8 are typical for floating-point parity after serialization.- Missing file should raise FileNotFoundError; corrupted files commonly raise ValueError/EOFError/OSError depending on backend — test accepts any of those to be robust.- For PyTorch use torch.save(model.state_dict()) and torch.load(...); compare model outputs after load similarly.

EasyTechnical

69 practiced

You implement padding and truncation logic for sequences with max_seq_len=128 for an NLP model. Write test cases that validate correct behavior for input lengths 0, 127, 128, and 129, including attention masks and special token placement. Describe off-by-one risks and how your tests catch them.

Sample Answer

Approach: assume pipeline does (1) tokenize -> token_ids (no special tokens), (2) add special tokens: [CLS] at index 0 and [SEP] at end, (3) truncate to max_seq_len=128, (4) pad to length 128 with PAD id 0, and (5) build attention_mask with 1 for non-PAD tokens. Below are pytest-style unit tests that validate behavior for input token lengths 0, 127, 128, 129 and check special token placement and attention masks.

python

import pytest

CLS_ID = 101
SEP_ID = 102
PAD_ID = 0
MAX_LEN = 128

# hypothetical function under test:
# def prepare_sequence(token_ids, max_len=MAX_LEN) -> (input_ids, attention_mask)
# - token_ids: list of token ids WITHOUT special tokens
# - returns input_ids length == max_len, attention_mask length == max_len

def assert_basic_properties(input_ids, attention_mask, nonpad_count):
    assert len(input_ids) == MAX_LEN
    assert len(attention_mask) == MAX_LEN
    # attention_mask sums to number of non-pad tokens
    assert sum(attention_mask) == nonpad_count
    # padded positions must be PAD_ID and have attention 0
    for i, m in enumerate(attention_mask):
        if m == 0:
            assert input_ids[i] == PAD_ID

def test_length_zero():
    # empty token list -> only CLS and SEP remain -> 2 tokens, rest PAD
    input_ids, attn = prepare_sequence([], MAX_LEN)
    assert input_ids[0] == CLS_ID
    assert input_ids[1] == SEP_ID
    assert input_ids[2] == PAD_ID
    assert_basic_properties(input_ids, attn, nonpad_count=2)

def test_length_127():
    # 127 tokens + CLS + SEP -> 129 -> must truncate to 128:
    # expected behavior: truncate one token so total==128 (CLS + 126 tokens + SEP)
    tokens = list(range(200, 327))[:127]  # dummy token ids
    input_ids, attn = prepare_sequence(tokens, MAX_LEN)
    assert input_ids[0] == CLS_ID
    assert input_ids[-1] == SEP_ID
    # number of non-pad tokens should be 128 (full)
    assert_basic_properties(input_ids, attn, nonpad_count=128)
    # ensure last token before SEP is original token (truncation preserved correct count)
    assert input_ids[-2] != PAD_ID

def test_length_128():
    # 128 tokens + CLS + SEP -> 130 -> must truncate to 128:
    # expected: keep CLS + 126 tokens + SEP (same as previous)
    tokens = list(range(200, 328))[:128]
    input_ids, attn = prepare_sequence(tokens, MAX_LEN)
    assert input_ids[0] == CLS_ID
    assert input_ids[-1] == SEP_ID
    assert_basic_properties(input_ids, attn, nonpad_count=128)

def test_length_129():
    # 129 tokens -> even more truncation; still result full length 128
    tokens = list(range(200, 329))[:129]
    input_ids, attn = prepare_sequence(tokens, MAX_LEN)
    assert input_ids[0] == CLS_ID
    assert input_ids[-1] == SEP_ID
    assert_basic_properties(input_ids, attn, nonpad_count=128)

Off-by-one risks and how tests catch them:- Counting special tokens: If implementation forgets to reserve slots for CLS/SEP, truncation may drop a real token or a special token. Tests assert CLS at idx 0 and SEP at idx -1 to catch misplacement.- Inclusive/exclusive truncation: Whether you truncate to keep total length == max_len including special tokens. Tests for input lengths 127,128,129 verify truncation logic by checking nonpad_count is exactly 128 and that the last token before SEP is not PAD.- Padding vs attention mask mismatch: Tests assert sum(attention_mask) equals nonpad tokens and that any attention_mask==0 corresponds to PAD_ID, catching off-by-one where mask/pad lengths misalign.- Zero-length input: ensures special tokens still present and padding starts at index 2, catching errors that assume at least one token exists.

These focused cases exercise boundary conditions and reveal off-by-one mistakes in counting tokens, special tokens, truncation, and mask construction.

EasyTechnical

68 practiced

You have a function normalize(arr: np.ndarray) -> np.ndarray that standardizes a numeric vector to zero mean and unit variance. Write two to three pytest unit tests in Python that validate behavior for: (1) an empty array, (2) a constant array (zero variance), and (3) an array containing NaN or Inf. State the expected result or exception for each case and show the test inputs and assertions.

Sample Answer

Below are three pytest unit tests for normalize(arr: np.ndarray) -> np.ndarray. Assumptions (stated in tests): - Empty array -> returns empty array (shape preserved).- Constant array (zero variance) -> raises ValueError to avoid division-by-zero.- Array with NaN or Inf -> raises ValueError for invalid inputs.

python

import numpy as np
import pytest
from mymodule import normalize  # replace with actual module

def test_normalize_empty_array():
    arr = np.array([], dtype=float)
    out = normalize(arr)
    assert isinstance(out, np.ndarray)
    assert out.size == 0
    assert out.shape == arr.shape

def test_normalize_constant_array_raises():
    arr = np.array([5.0, 5.0, 5.0])
    with pytest.raises(ValueError):
        normalize(arr)

def test_normalize_with_nan_or_inf_raises():
    arr_nan = np.array([1.0, np.nan, 3.0])
    arr_inf = np.array([1.0, np.inf, 3.0])
    with pytest.raises(ValueError):
        normalize(arr_nan)
    with pytest.raises(ValueError):
        normalize(arr_inf)

Notes:- If your expected behavior differs (e.g., return zeros for constant arrays), adjust assertions accordingly.- These tests ensure safe handling of edge cases common in ML data pipelines.

MediumTechnical

80 practiced

You must follow Test-Driven Development (TDD) for a new scoring microservice. Produce an ordered list of tests you would write before implementation, including unit tests for preprocessors, schema/contract tests, mocked inference tests, latency SLAs, and integration tests with the feature store. Explain why you chose the order and what constitutes a passing vs failing test.

Sample Answer

1) Unit tests — preprocessor functions (tokenization, normalization, missing-value imputation, feature engineering) - Why first: fastest feedback, isolates deterministic data transforms. - Passing: given fixed input, output exactly matches golden fixtures (including types and ranges). - Failing: output mismatch, exceptions, or non-deterministic outputs.

2) Schema/contract tests for request/response and feature payloads (JSON Schema/Avro/Protobuf) - Why second: defines API boundary early so downstream code and CI can validate contracts. - Passing: sample messages validate against schema; required/optional fields and types enforced. - Failing: schema violations or ambiguous/unstable fields.

3) Unit tests for model wrapper (serialization/deserialization, input mapping) with mocked model object - Why third: ensures the serving wrapper behaves before real model dependency. - Passing: wrapper calls model.predict with mapped inputs, handles model exceptions, and returns expected shapes. - Failing: wrong input mapping, unhandled exceptions, incorrect output formatting.

4) Mocked inference integration tests (end-to-end serving path with a fake model) - Why fourth: verifies full runtime wiring (preprocessor → wrapper → postprocessor) without heavy model compute. - Passing: given test inputs, service returns expected scores, correct HTTP codes, and logs. - Failing: incorrect end-to-end flow, serialization issues, or error responses.

5) Latency and throughput SLA unit/integration tests (synthetic load tests in CI with mocked and small real model) - Why fifth: verify non-functional requirements before full feature-store coupling. - Passing: p95/p99 latency under SLA, acceptable CPU/memory within resource limits. - Failing: SLA breaches or resource exhaustion.

6) Contract tests with feature store (schema compatibility, feature availability, backfill behavior) - Why sixth: ensures features produced by store match expected schema/timeliness. - Passing: features retrieved for test entities match types, timestamps, and freshness windows. - Failing: missing features, type mismatches, stale timestamps.

7) End-to-end integration tests with real model and real feature store in staging (smoke + regression with representative data) - Why last: slowest and most brittle — run after all smaller tests guarantee system readiness. - Passing: end-to-end accuracy/regression thresholds met, feature freshness OK, no exceptions, and latency within SLA. - Failing: prediction drift, missing features, pipeline failures, or SLA breaches.

Order rationale summary: go from fastest, most deterministic, low-cost checks (unit, schema) to broader integration and non-functional tests to get quick feedback, catch design issues early, and reduce flakiness. Each test should be runnable in CI with clear assertions and deterministic fixtures; failing tests must block merges until fixed.

Unlock Full Question Bank

Get access to hundreds of Edge Case Identification and Testing interview questions and detailed answers.

Join thousands of developers preparing for their dream job.