Test Case Design and Edge Case Analysis Questions

Direct answer
A fuzz-testing harness for an ML (machine learning) inference API applies the same generic edge-case-discovery technique as parser fuzzing (mutation, coverage feedback, crash triage) to the API's REQUEST SURFACE rather than its model weights, targeting three failure classes the question names explicitly: hard crashes (the process dies), hangs (a request that never returns, e.g. a pathological input driving an unbounded loop in preprocessing), and logical failures (the request completes and returns 200, but produces a nonsensical or unsafely-formed response), using malformed JSON, extra fields, very long strings, extreme numeric values, and wrong types as the mutation dimensions.

Structured elaboration

• Generation vs mutation fuzzers: a mutation-based fuzzer starts from real, valid request payloads (captured from production logs or hand-written examples) and perturbs them; a generation-based fuzzer builds requests from a schema (e.g. the API's OpenAPI/JSON-schema definition) without needing real seed examples, which is the more practical starting point for an inference API, because the request schema is usually well-defined even before real traffic exists, and generation naturally produces the "extra fields" and "wrong types" cases the question names by construction (generate a request, then add an undeclared field, or substitute a field's declared type for an incompatible one).
• Mutation strategies (bit flips, boundary values) mapped onto the five named request-shape categories: malformed JSON is produced by bit-flipping a byte inside an otherwise valid payload, or by truncating it at a random offset, or by corrupting a structural character (a brace, a comma); extra fields are produced by injecting undeclared keys into an otherwise valid payload, testing whether the API's deserializer strictly rejects unknown fields or silently (and possibly dangerously) passes them through to downstream logic; very long strings and extreme numeric values are boundary-value mutations applied specifically to fields the model consumes (e.g. a text field mutated to megabytes of repeated tokens, or a numeric feature field mutated to 1e308, -1, 0, or NaN-equivalent JSON representations); wrong types substitutes a string where a number is expected and vice versa, which is the case most likely to reveal whether the API validates against its schema BEFORE invoking the model or lets a type-confused value flow into featurization code that assumes a specific type.
• Measuring coverage of code paths: instrument the request-handling and preprocessing/featurization code (not the model's internal weights, which are not meaningfully "covered" by input fuzzing in the traditional sense) with the same line/branch coverage tooling used for any Python service (e.g. coverage.py), and track which validation and preprocessing branches a growing corpus of mutated requests has exercised; a mutation corpus that only ever reaches the "happy path" featurization code and never the type-mismatch or out-of-range handling branches indicates the fuzzer's mutation strategy needs to be more aggressive on those specific fields, not that the API is well-tested.
• Reducing flakiness in the harness itself (a harness-design concern here, not diagnosing an already-flaky test suite): pin the inference call's non-determinism sources before fuzzing (fix random seeds in any stochastic decoding path, disable GPU-nondeterministic kernels or pin to CPU execution for the harness run, and add an explicit request timeout so a hang is detected deterministically rather than the harness itself hanging indefinitely), so that a crash or hang found by the fuzzer reproduces reliably when replayed, which is a prerequisite for the crash to be triage-able at all.
• CI (continuous integration) integration: run a short, time-boxed fuzzing pass (a fixed number of mutated requests, not a fixed wall-clock duration, so the check is reproducible across CI runners of different speeds) against every change to the request-handling or featurization code, and separately run the full campaign continuously as a longer-lived background job rather than gating every commit, mirroring how coverage-guided fuzzing is normally integrated for any parser or request-handling surface.

Worked example: malformed JSON feature payload
For a concrete inference endpoint accepting {"features": {"age": 34, "income": 52000.0, "region": "west"}}, the mutation corpus would include: {"features": {"age": "34"}} (wrong type: string where a number is expected, testing whether the schema validator rejects it before it reaches feature encoding); {"features": {"age": 34, "income": 52000.0, "region": "west", "__proto__": "x"}} (an extra, potentially dangerous field name, testing that unknown keys are rejected or ignored rather than passed to an unsafe deserializer); {"features": {"age": 999999999999, "income": -1e308}} (extreme numeric values, testing whether featurization clamps, rejects, or silently produces a garbage prediction on out-of-distribution numeric input); and a truncated payload {"features": {"age": 34, "inc (malformed JSON, testing that the API returns a clean 400-class error rather than a 500 or a hang). Each of these targets a different point in the request pipeline: schema validation, key allow-listing, feature-range validation, and JSON parsing itself, so a single fuzzing pass that only exercises one of these four is not exercising the full pipeline the question asks about.

Trade-offs and pitfalls
The most common wrong turn is fuzzing only the JSON structure (malformed JSON, extra fields) and never the semantic content of well-formed-but-extreme values, which misses the logical-failure class the question explicitly names: a request that is syntactically perfect JSON with a region value the model was never trained on, or an income value far outside the training distribution, will not crash or hang, but can produce a confidently wrong prediction, which is arguably the most dangerous of the three failure classes because nothing in the API's error handling signals it. The other pitfall is fuzzing the LIVE model in a way that is not reproducible (letting decoding randomness vary run to run), which makes every "logical failure" finding impossible to confirm was actually caused by the mutated input rather than ordinary model stochasticity; pinning determinism in the harness, as described above, is a prerequisite for the whole exercise to produce trustworthy findings.

Direct answer
Integer overflow (a result exceeding a fixed-width type's max representable value) and underflow (going below its min, or in a different sense, a floating-point value rounding to zero) show up in AI systems anywhere a fixed-width counter, index, or accumulator is used inside training or inference. Four concrete places it bites: 8-bit quantized accumulators, timestamp arithmetic in event/telemetry pipelines, bucketed histogram counters, and embedding-table row indices, each needs its own targeted test, not one generic "check for overflow" test.

Structured elaboration

1. 8-bit quantized accumulators. Quantized inference kernels represent weights and activations as int8 to save memory and compute, but the accumulator summing many int8 x int8 products must be wide enough to hold the sum without wrapping; using an int8 accumulator (rather than the int32 real kernels use) silently corrupts the output the moment enough terms accumulate. A multiply-accumulate (MAC) operation done in a too-narrow accumulator is the textbook version of this bug.
2. Timestamp arithmetic. A monotonic millisecond counter stored as a signed 32-bit integer wraps after 2^31 milliseconds, about 24.9 days. Any service that truncates a 64-bit epoch-millisecond timestamp down to 32 bits (to save space, or because of a legacy schema) will wrap on that cadence; the real danger is not the wrap itself but a different downstream service re-widening the already-wrapped value to 64 bits and subtracting, producing a nonsensical multi-billion-millisecond "negative latency."
3. Bucketed histogram counters. A per-bucket counter in a training-metrics or feature-frequency histogram, if stored as an unsigned 32-bit integer, wraps back to 0 after 4,294,967,296 increments to that bucket; for a high-frequency feature or token bucket in a large-scale pipeline, that ceiling is reachable, and the counter silently resets rather than erroring.
4. Embedding-table row indices: a vocabulary or entity-embedding table with more than 2^31 - 1 rows cannot be addressed by a signed 32-bit row index at all; any lookup past that row simply cannot be expressed, not just miscomputed. This is less "arithmetic overflow of a running value" and more "the index type's range is smaller than the address space it needs to cover," but it is the same fixed-width-type root cause.

Worked example (executed)
Simulating case 1, an int8-accumulated dot product of two 64-element int8 vectors versus an int32-accumulated one, against an exact (unbounded) reference sum:

import numpy as np
rng = np.random.default_rng(seed=42)
w = rng.integers(-100, 100, size=64, dtype=np.int8)
a = rng.integers(-100, 100, size=64, dtype=np.int8)
products = w.astype(np.int32) * a.astype(np.int32)
true_sum = int(products.sum())

acc8 = np.int8(0)
for p in products:
    acc8 = np.int8(acc8 + np.int8(p))  # deliberately narrow accumulator

acc32 = np.int32(0)
for p in products:
    acc32 = np.int32(acc32) + np.int32(p)

print(f"true_sum={true_sum}, acc32={int(acc32)}, acc8={int(acc8)}")

Output: true_sum=-4029, acc32=-4029, acc8=67 (acc32 matches the exact reference exactly; acc8, the deliberately narrow accumulator, is wrong by construction because it wrapped multiple times during the loop).

Case 2, timestamp arithmetic:

INT32_MAX = 2147483647
print("days to wrap:", INT32_MAX / 86400000)

def trunc32(x):
    x = x % (2**32)
    return x - 2**32 if x >= 2**31 else x

t1_true, t2_true = 2147483642, 2147483657  # true 64-bit ms, 15ms apart, straddling the wrap
t1, t2 = trunc32(t1_true), trunc32(t2_true)
print(f"t1_trunc={t1}, t2_trunc={t2}, naive downstream diff={t2 - t1}, true diff={t2_true - t1_true}")

Output: days to wrap: 24.85513480324074 (a 32-bit millisecond counter overflows after about 24.9 days); t1_trunc=2147483642, t2_trunc=-2147483639, naive downstream diff=-4294967281, true diff=15, confirming a downstream service that naively re-widens the already-truncated values to 64-bit and subtracts gets a nonsense multi-billion-millisecond negative latency instead of the true 15ms gap.

Case 4, embedding index:

INT32_MAX = 2147483647
vocab_size = 2**31 + 100
last_idx = vocab_size - 1
print(f"vocab_size={vocab_size}, last_idx={last_idx}, exceeds INT32_MAX={last_idx > INT32_MAX}")

Output: vocab_size=2147483748, last_idx=2147483747, exceeds INT32_MAX=True, confirming a table with more than 2**31 - 1 rows has a last valid row index that cannot be represented by a signed 32-bit index at all.

Case 3, bucketed histogram counter: simulating an unsigned 32-bit counter (UINT32_MAX = 4294967295) that starts 2 increments away from its ceiling (start = 4294967293) and receives 5 more increments, tracked both as an exact (unbounded) count and as a wrapped % 2**32 value:

UINT32_MAX = 2**32 - 1
start = UINT32_MAX - 2  # 4294967293
true_count = start
sim_count = start % (2**32)
for _ in range(5):
    true_count += 1
    sim_count = (sim_count + 1) % (2**32)
print(f"true_count={true_count}, sim_count={sim_count}")

Output: true_count=4294967298, sim_count=2. The exact count and the wrapped counter diverge by exactly one full wraparound (4294967298 - 2 = 4294967296 = 2**32), and critically the wrapped counter's value (2) looks like a perfectly ordinary small, healthy count, nothing about it signals that a wrap happened; the test's pass/fail criterion has to be "does the wrapped counter equal the exact reference count," not "does the wrapped counter look reasonable," since a wrapped counter always looks reasonable on its own.

Trade-offs & pitfalls
The fix for cases 1-3 is the same pattern every time: widen the accumulator/counter type relative to what it accumulates, not relative to what a single input value needs; the bug is specifically that a single int8 value fits fine, but the running sum of many of them does not. For timestamps specifically, the safest fix is to never truncate the wide timestamp in the first place (keep 64-bit end to end) rather than truncating and hoping downstream code always re-derives durations correctly across a possible wrap, that hope is exactly what fails in practice. Tests for all of these should use an exact, unbounded reference (Python's arbitrary-precision integers, or a wider accumulator type) as the oracle to diff against, a spot-check on "normal-sized" inputs will never trigger any of these, only inputs deliberately chosen to sit near or past the type's boundary will.

Direct answer
Model prediction functions fail differently at inference time than at training time, because a serving path has to handle whatever a caller sends, not a curated training set: empty batches, single-sample batches, duplicated inputs, oversized batches, all-NaN (Not a Number) inputs, and extreme class imbalance in the OUTPUT each need an explicit, tested response, and several of them (single-sample batches especially) hide subtle shape bugs that a test suite built only around normal-sized batches will never exercise.

Structured elaboration

Worked example (executed)

import numpy as np

def predict_batch(model_fn, X, max_batch_size=10_000):
    if X.shape[0] == 0: raise PredictionInputError("empty_dataset")
    if X.shape[0] > max_batch_size: raise PredictionInputError("batch_too_large")
    if np.all(np.isnan(X)): raise PredictionInputError("all_nan_input")
    if X.ndim == 1: X = X.reshape(1, -1)
    preds = model_fn(X)
    n_unique_rows = len(np.unique(X, axis=0))
    duplicate_fraction = 1 - (n_unique_rows / X.shape[0])
    pred_classes, counts = np.unique(preds, return_counts=True)
    max_class_fraction = counts.max() / counts.sum()
    return {"predictions": preds, "duplicate_input_fraction": round(float(duplicate_fraction), 4),
            "max_predicted_class_fraction": round(float(max_class_fraction), 4),
            "extreme_class_imbalance_flag": bool(max_class_fraction > 0.95)}

Run against a dummy threshold model (predicts class 1 if a row's sum is positive, else class 0):

normal_batch (3 distinct rows): duplicate_input_fraction=0.0, max_predicted_class_fraction=0.6667, imbalance_flag=False
empty_dataset: REJECTED, reason=empty_dataset
single_sample_2d [[1.0, 2.0]]: OK, max_predicted_class_fraction=1.0, imbalance_flag=True
single_sample_1d [1.0, 2.0] (reshaped): OK, identical result to the 2-D single-sample case above
duplicated_inputs (5 identical rows): duplicate_input_fraction=0.8, max_predicted_class_fraction=1.0, imbalance_flag=True
batch_too_large (10,001 rows): REJECTED, reason=batch_too_large
all_nan_input: REJECTED, reason=all_nan_input
extreme_class_imbalance (99 rows class 0, 1 row class 1): duplicate_input_fraction=0.98, max_predicted_class_fraction=0.99, imbalance_flag=True

The single-sample results reveal a genuine pitfall worth naming explicitly: max_predicted_class_fraction is trivially 1.0 for ANY single-sample batch, since there is only one prediction and it is necessarily 100% of the batch. The imbalance flag firing here is not a bug in the function, it is a property of the metric itself being degenerate at n=1, and a monitoring dashboard that alerts on this metric needs to either exclude single-sample batches from that specific check or interpret it with that caveat, rather than treating every single-sample "imbalance" alert as meaningful.

Trade-offs and pitfalls
The most common wrong turn is testing only "normal" batch sizes (say, 32 to 256) and never batch size 1, which is exactly where the reshape/squeeze bug hides and where, as shown above, an imbalance metric becomes meaningless without being obviously broken. A second pitfall is checking for all-NaN only, and never a PARTIALLY NaN row (one bad feature among otherwise-valid ones), which is arguably the more common real-world case and needs its own explicit policy decision (reject the row, impute, or pass through and let the model handle it, if it can). A third is treating the class-imbalance check as a hard gate that rejects the batch, when it is genuinely a monitoring signal in most contexts, a naturally imbalanced real batch (e.g. fraud detection, where legitimate transactions vastly outnumber fraudulent ones) should not be rejected outright just because a naive threshold check treats normal skew as a bug.

Direct answer
Multilingual tokenization has to be validated against Unicode's own structural edge cases, not just "does it handle non-English text": a Zero-Width Joiner (ZWJ, the invisible character U+200D that fuses adjacent codepoints into one visual and logical unit) sequence, a combining-character sequence that is codepoint-different but visually and semantically identical to its precomposed form, and right-to-left (RTL) script text mixed with left-to-right content, each breaks a different naive assumption a tokenizer might make (that one codepoint is one character, that visually-identical strings are codepoint-identical, and that token order matches display order).

Structured elaboration
The unit under test in all three cases is really the grapheme cluster, the user-perceived character, which can span multiple Unicode codepoints. A tokenizer that operates on raw codepoints (Python's list(some_string), or naive UTF-16 code-unit indexing in JavaScript) will frequently split a single grapheme cluster into several tokens, which is wrong for any tokenizer whose contract is "one token boundary per meaningful unit."

Test case 1: ZWJ emoji sequence
Input: the "family" emoji, encoded as man + ZWJ + woman + ZWJ + girl (4 codepoints joined into one visual glyph). Expected behavior: a grapheme-aware tokenizer treats the whole joined sequence as one token; a naive codepoint-level tokenizer splits it into multiple separate codepoints.
Assertion in an automated harness: assert family_emoji in tokens and tokens.count(family_emoji) == 1 for a single occurrence.

Test case 2: combining vs. precomposed characters
Input: the same visual word ("café") represented two different ways, once with the precomposed codepoint U+00E9 (e-acute as a single codepoint) and once with the decomposed sequence 'e' + U+0301 (a base letter plus a combining acute accent, two codepoints). Expected behavior: the two raw strings are codepoint-different (so a naive == string comparison says they differ), but Unicode Normalization Form C (NFC), the standard "always fully compose combining sequences" canonical form, resolves both to the identical string, and the combining mark must fuse onto its base letter as a single grapheme cluster, not form a token of its own.
Assertion: unicodedata.normalize('NFC', precomposed) == unicodedata.normalize('NFC', decomposed).

Test case 3: RTL Hebrew text plus surrogate pairs
Input: a Hebrew word (RTL script) mixed with English text (LTR) and a ZWJ-joined emoji containing a variation selector (a modifier codepoint that changes how the preceding character renders). Expected behavior: tokens are produced in logical order (the order the text is read/typed in, left-to-right through the underlying character stream) regardless of how a renderer visually displays the RTL portion, and the emoji sequence still collapses to one grapheme cluster despite spanning three codepoints. Surrogate pairs (the two-code-unit encoding UTF-16 uses to represent a codepoint above U+FFFF, such as most emoji) are the concrete mechanism by which a naive JavaScript tokenizer breaks this: JavaScript strings are UTF-16 internally, and indexing or slicing by .length operates on 16-bit code units, not codepoints, so a naive slice can cut a surrogate pair in half, producing an invalid, unpaired code unit.
Assertion: tokens[0] == hebrew_word (intact, first, in logical order) and len(grapheme_clusters(emoji)) == 1.

Worked example (executed)
A minimal grapheme-cluster tokenizer (fusing Unicode combining marks, ZWJ, and variation selectors onto the preceding cluster), run against all three cases:

import unicodedata

ZWJ = '\u200d'
VS16 = '\ufe0f'

def is_mark(ch):
    return unicodedata.category(ch) in ('Mn', 'Mc', 'Me')

def grapheme_clusters(s):
    clusters, current, i, n = [], '', 0, len(s)
    while i < n:
        ch = s[i]
        if current == '':
            current = ch; i += 1; continue
        if is_mark(ch) or ch == VS16 or ch == ZWJ or s[i-1] == ZWJ:
            current += ch; i += 1; continue
        clusters.append(current); current = ch; i += 1
    if current:
        clusters.append(current)
    return clusters

def simple_tokenize(s):
    tokens, current = [], ''
    for cluster in grapheme_clusters(s):
        if cluster.strip() == '':
            if current:
                tokens.append(current); current = ''
        else:
            current += cluster
    if current:
        tokens.append(current)
    return tokens

# Test 1: ZWJ emoji sequence
family = '\U0001F468' + ZWJ + '\U0001F469' + ZWJ + '\U0001F467'  # man ZWJ woman ZWJ girl
phrase1 = f"our {family} went to the park"
print(f"Test1: phrase has {len(phrase1)} raw codepoints; family = {len(family)} codepoints "
      f"-> {len(grapheme_clusters(family))} cluster(s); tokens={simple_tokenize(phrase1)}")

# Test 2: combining vs precomposed
precomposed, decomposed = 'café', 'cafe' + '\u0301'
print(f"Test2: precomposed={len(precomposed)} codepoints, decomposed={len(decomposed)} codepoints, "
      f"raw equal={precomposed == decomposed}, "
      f"NFC equal={unicodedata.normalize('NFC', precomposed) == unicodedata.normalize('NFC', decomposed)}, "
      f"decomposed clusters={grapheme_clusters(decomposed)}")

# Test 3: RTL Hebrew + LTR + ZWJ emoji with variation selector
health_worker = '\U0001F469' + ZWJ + '\u2695' + VS16  # woman ZWJ staff-of-aesculapius VS16
hebrew_word = 'עברית'
phrase3 = f"{hebrew_word} and English mixed with {health_worker}"
tokens3 = simple_tokenize(phrase3)
print(f"Test3: health_worker={len(health_worker)} codepoints -> {len(grapheme_clusters(health_worker))} "
      f"cluster(s); tokens={tokens3}; hebrew word intact and first={tokens3[0] == hebrew_word}")

Output:

Test1: phrase has 26 raw codepoints; family = 5 codepoints -> 1 cluster(s); tokens=['our', '👨‍👩‍👧', 'went', 'to', 'the', 'park']
Test2: precomposed=4 codepoints, decomposed=5 codepoints, raw equal=False, NFC equal=True, decomposed clusters=['c', 'a', 'f', 'é']
Test3: health_worker=4 codepoints -> 1 cluster(s); tokens=['עברית', 'and', 'English', 'mixed', 'with', '👩‍⚕️']; hebrew word intact and first=True

Test 3's health-worker emoji is woman + ZWJ + medical symbol + variation selector, which is 4 codepoints (not 3: each of the four named components is its own codepoint), collapsing to exactly 1 grapheme cluster, the same fusion behavior as the family emoji in test 1.

Separately confirmed the JavaScript surrogate-pair mechanism named in test case 3 (Node.js, same family emoji): family.length (UTF-16 code units) is 8, [...family].length (codepoint-aware iteration) is 5, and a naive family.slice(0, 1) returns a single lone, unpaired surrogate code unit ("\ud83d"), not a valid character, confirming that naive UTF-16-indexed tokenization genuinely corrupts the sequence rather than being a hypothetical risk.

Complexity and edge cases
The grapheme-cluster tokenizer is O(n) time and O(n) space in the number of input codepoints, one linear pass classifying each codepoint's Unicode category. Edge cases: a multi-codepoint ZWJ-joined emoji sequence, a combining-mark sequence versus its precomposed equivalent, a variation-selector-modified emoji, RTL script text mixed with LTR text, and a plain whitespace-delimited word with no special Unicode structure at all (the case that must NOT be over-clustered).

Trade-offs & pitfalls
A hand-rolled grapheme-cluster approximation (as shown, fusing combining marks and ZWJ sequences) is good enough to catch the specific bugs above, but it is not a full UAX #29 (the Unicode Text Segmentation standard) implementation, real production tokenizers should use a vetted Unicode-segmentation library rather than reimplementing the full algorithm, which also has to handle Hangul conjoining jamo, regional-indicator flag pairs, and extended pictographic sequences correctly. A common mistake is testing only ASCII and one "obviously foreign" script (like testing English and Chinese and calling it done); combining characters, ZWJ sequences, and RTL text are each independent axes of risk that a single non-Latin test script does not automatically cover. For subword tokenizers specifically (BPE, WordPiece), a grapheme cluster surviving intact through pre-tokenization does not guarantee the subword vocabulary handles it gracefully, a rare-script grapheme cluster with no vocabulary coverage still degrades to a byte-level or unknown-token fallback, which is a distinct, separately-testable failure mode from the segmentation bugs covered here.

Direct answer
The riskiest boundary in padding/truncation logic sits exactly where special tokens meet the length budget: if an implementation truncates content to max_seq_len and then adds [CLS]/[SEP] on top, every sequence quietly overshoots the model's real input limit by 2 tokens. The four required test lengths (0, 127, 128, 129) look like they probe "at, just under, just over max_seq_len," but the actual content budget is max_seq_len - 2 = 126, so all of 127, 128, and 129 exceed it, and a correct implementation must truncate content identically (to 126 tokens) for all three while a buggy one will not.

Structured elaboration
Using a BERT-style scheme with a [CLS] token at the start and a [SEP] token at the end: content_budget = max_seq_len - 2. The correct sequence is built as [CLS] + content[:content_budget] + [SEP], then padded with a pad token up to max_seq_len if shorter. The attention mask must be 1 for every real token including [CLS] and [SEP] (not just the content tokens), and 0 for every padding position.

The off-by-one risk is specifically in how content_budget is derived: a naive implementation that checks if len(tokens) <= max_seq_len and only then decides whether to truncate, without ever subtracting the 2 reserved slots, will treat an input of length 127 as "fits" (since 127 <= 128) when it actually still needs to lose at least 1 token to leave room for both special tokens. A second, subtler risk is off-by-one in the subtraction itself (using max_seq_len - 1 instead of -2, accounting for only one special token), which silently drops one extra content token from every sequence.

Worked example (executed; the original draft shipped only the two function definitions with the results table narrated in prose and no driver code actually producing it, and the function bodies were flattened to a single indent level, both fixed below with a real driver loop whose printed output matches the table)

CLS_ID, SEP_ID, PAD_ID, MAX_SEQ_LEN = 101, 102, 0, 128

def pad_truncate_fixed(token_ids, max_seq_len=MAX_SEQ_LEN):
    content_budget = max_seq_len - 2
    content = token_ids[:content_budget]
    seq = [CLS_ID] + content + [SEP_ID]
    attention_mask = [1] * len(seq)
    pad_needed = max_seq_len - len(seq)
    seq = seq + [PAD_ID] * pad_needed
    attention_mask = attention_mask + [0] * pad_needed
    return seq, attention_mask

def pad_truncate_buggy(token_ids, max_seq_len=MAX_SEQ_LEN):
    # BUG: truncates to max_seq_len BEFORE adding the 2 special tokens
    content = token_ids[:max_seq_len]
    seq = [CLS_ID] + content + [SEP_ID]
    return seq, [1] * len(seq)

for length in [0, 127, 128, 129]:
    tokens = list(range(1000, 1000 + length))
    fseq, fmask = pad_truncate_fixed(tokens)
    bseq, bmask = pad_truncate_buggy(tokens)
    content_kept = min(length, MAX_SEQ_LEN - 2)
    content_dropped = length - content_kept
    print(f"len={length}: fixed_total={len(fseq)} fixed_mask_sum={sum(fmask)} "
          f"kept/dropped={content_kept}/{content_dropped} buggy_total={len(bseq)}")

Running both implementations against input lengths 0, 127, 128, and 129 (via the driver loop above) produced:

The fixed implementation always produces exactly 128 tokens with [CLS] at position 0, [SEP] immediately after the last real content token, and the attention mask matching real-token count exactly (2 for the empty input, since only [CLS]/[SEP] are real; 128 for the other three, since the content budget is fully used and no padding remains). The buggy implementation produces a 2-token sequence for empty input (itself fine) but overshoots max_seq_len by 1 token at input length 127 and by 2 tokens at 128 and 129, exactly the off-by-one/off-by-two the reserved-slots bug predicts, and exactly what the four chosen test lengths are designed to expose: the length-127 case alone already proves the bug, since a correct implementation caps at 128 total regardless.

Trade-offs & pitfalls
Teams often special-case the empty-input case inconsistently, for example skipping [CLS]/[SEP] entirely when there is no content, which then breaks any downstream code that assumes position 0 is always [CLS]; the test at input length 0 exists specifically to pin down that special tokens are still present even with zero real content, as shown above (total length 128, mask sum 2, not 0). A second pitfall is testing only the attention mask's SUM (a scalar) rather than its exact positions: a mask that has the right total count of 1s but marks the wrong positions (for example, marking padding as attended while missing [SEP]) would pass a sum-only assertion while still being wrong; the test in the worked example checks that [SEP] sits at the boundary between real tokens and padding, not just that the mask has the right count. Finally, truncation direction is a real design decision, not a given: dropping from the end of the content (as shown here) is standard for tasks where early context matters most, but a task where the END of a long document matters more (for example, a document's conclusion) would need the opposite truncation direction, and that choice deserves its own explicit test rather than being assumed.