Lyft Senior QA Engineer Interview Preparation Guide

Situation: On one project, engineering and design were operating in separate lanes, which caused late feedback and rework.

Task: I needed to rebuild trust and unblock delivery without turning the problem into a blame conversation.

Action: I set up joint working sessions where both teams reviewed the same problem statement and success criteria. I also introduced a shared definition of done so we were clear about what “ready” meant before handoff. To build trust, I made sure both sides had equal airtime, captured decisions in writing, and followed through on small commitments quickly. After that, I kept the collaboration healthy with regular check-ins, shared demos, and a single place to track open questions.

Result: The teams started catching issues earlier, handoffs became smoother, and there was less tension around ownership. The biggest lesson was that silos break down faster when people share context and make small reliable commitments over time.

Approach overview
I treat traceability as a living, automated mapping between requirements → test cases → execution results → defects. That ensures every defect can be traced back to the test(s) that found it and vice‑versa.

Naming / ID conventions

• Tests: TC-<REQID>-<seq> (e.g., TC-REQ123-05) so requirement linkage is implicit.
• Defects: DEF-<PROJECT>-<YYYYMMDD>-<seq> or use tracker auto-ID (e.g., PROJ-456).
• Automation: mirror test IDs as script names or tags (e.g., test_TC-REQ123-05).

Linking strategies

• Use test management system (TMS) features to link test cases to requirements and to Jira/GitHub issues; always add the defect ID in the test execution record.
• In the issue tracker, add a “Test Cases” field or comment with TC IDs and a direct TMS link; use webhooks to sync status updates both ways.
• For automated runs, include defect references in test metadata so CI reports contain links.

Marking for regression

• When a bug is fixed, update the affected test(s) with labels: regression=yes, priority=high, and add to the “Critical Regression” suite in the TMS. Automations should pick suites by tag.

How mapping aids release & audits

• Release: I produce a defect-to-test matrix showing which defects are covered by regression tests and which tests passed post-fix — this informs go/no-go.
• Audits: The linked history (test execution records, defect lifecycle, and requirement IDs) provides evidence of verification and compliance. It also speeds root-cause and trend analysis.

Outcome: consistent IDs, bi-directional links, automated syncing, and clear regression tagging make decisions and audits traceable and defensible.

Situation & goal
Design an A/B experiment to test whether targeted regression selection (TRS) reduces defect escape rate versus prior broad-regression strategy.

Treatment & control

• Control: teams use prior strategy (full or risk-agnostic regression).
• Treatment: teams apply TRS (selecting test subsets based on risk/impact/changed code).
• Randomize at release-build or team-sprint level to avoid cross-contamination.

Metrics to collect

• Primary: defect escape rate = (production defects originating from release) / (total defects discovered pre+post release).
• Secondary: mean time to detect (MTTD) in prod, severity-weighted escapes, test execution time, cycle time, false-negative rate of selected regressions.
• Instrument: bug tracker tags, CI build IDs, deployment timestamps.

Sample size / duration

• Estimate baseline escape rate p0. Choose minimum detectable effect (e.g., 20% relative reduction). Use two-proportion sample-size formula to compute required releases per arm; if uncertain, run for minimum 6–8 releases or 8–12 weeks to capture variability.

Statistical test

• Two-proportion z-test (or Fisher’s exact if small counts) on escape rates. Use 95% CI, report p-value and effect size (absolute and relative reduction). Adjust for multiple comparisons if testing many cohorts.

Confounders & interpretation

• Monitor covariates: release size, number of changes, team experience, test automation coverage. Use stratified randomization or regression adjustment (logistic regression) to control confounders.
• If imbalance remains, report adjusted effect and perform sensitivity analysis (e.g., E-value) to estimate robustness to unmeasured confounding.
• Conclude effectiveness if statistically significant reduction and operational metrics (e.g., no large increase in missed high-severity defects or unacceptable cycle-time changes).

Direct answer

Teaching a skill you don't have natural talent for means separating what you know intuitively from what's actually teachable. You diagnose the real gap first, build an explicit, decomposed framework for the skill (even though you perform it by feel), and validate progress by watching the person apply it independently, not by how confident the coaching sessions felt.

Approach to teaching outside your natural strength

Diagnose before prescribing. "Struggles with X" is rarely one problem. Watch or review their actual attempt and separate the layers: is it a knowledge gap (they don't know the structure), a delivery gap (they know the structure but execution is shaky), or a confidence gap (they know it and can do it, but freeze under real stakes). Each needs a different intervention.

Decompose your own tacit skill into explicit steps. If you're good at something without having consciously learned it as a framework, you have to reverse-engineer your own process before you can teach it. Skipping this step and just saying "do what feels right" doesn't transfer anything.

Practice at graduated, increasing stakes. Start with low-stakes reps where mistakes are cheap and recoverable, then move toward the real, higher-stakes version. Jumping straight to the real thing conflates skill-building with performance evaluation in the person's head, which raises anxiety and slows learning.

Give feedback on the mechanism, not just the outcome. "That worked" or "that didn't work" is much less useful than pointing at which specific move in their approach caused the result.

Worked example

Situation: someone you're mentoring is excellent at the core technical work but has a real gap in a skill that doesn't come naturally to you either, say, communicating findings clearly to people outside the immediate team. Their material was always technically sound, but reviews ran long and the point often got lost.

Task: help them close that gap over a defined stretch, without pretending you have natural talent for it yourself.

Action: you watched a recording of one of their sessions together and separated content problems (no clear headline, too much detail up front) from delivery problems (pace, not anticipating pushback). You gave them a simple structure to practice against: state the conclusion first, then the supporting evidence, then the recommendation. You ran a couple of low-stakes rehearsals where you played a skeptical stakeholder, then let them run the real session solo.

Result: over a few sessions, their reviews needed fewer clarifying follow-up questions from the room, and the structure started showing up unprompted in written material too, not just live presentations. The real signal wasn't how the coaching sessions felt: it was watching them handle a session you weren't part of and hearing secondhand that it landed cleanly.

Trade-offs and pitfalls

A common junior-mentor mistake is trying to transfer your own tacit competence directly ("just do what I do") instead of decomposing it. That fails specifically because the skill you're teaching is one you never consciously learned as steps.

Another mistake: avoiding coaching on gaps you don't personally excel at, on the theory you're not qualified. You don't need to be naturally gifted at a skill to teach its structure. You need to be willing to build the explicit framework, which sometimes non-naturals do better than naturals, because they had to learn it deliberately themselves.

The real trade-off is time. Teaching a skill outside your own strength takes longer to prepare for, because you can't rely on instinct in the room. That prep time is where the actual coaching value gets built.

Direct answer

Acceptance test cases for adding OAuth 2.0 login need to cover the happy path, the specific error states a client will actually encounter, and the edge cases around consent and token lifecycle that are easy to forget precisely because they only show up after the initial integration is already working.

Structured elaboration

Three credentials show up throughout the cases below and are easy to conflate: the authorization code is a short-lived, single-use value the authorization server hands back after the user approves access, its only job is to be exchanged for real credentials; the access token is what the app then uses on every API call to act on the user's behalf, and it's normally short-lived too; the refresh token is a longer-lived credential the app stores and uses to get a new access token later without asking the user to log in again. With that vocabulary in place:

Happy path.

• A third-party app redirects a user to authorize, the user approves, and the app receives a valid authorization code.
• The app exchanges that code for an access token and (if applicable) a refresh token.
• The app successfully calls a protected endpoint on the user's behalf using the access token.

Error states.

• The user denies consent at the authorization screen: the app should receive a clear denial response, not an ambiguous error.
• An invalid or already-used authorization code is presented for exchange: expect a clear rejection, not a silently-issued token.
• A third-party app requests scopes it isn't registered for: expect the request to be rejected before any token is issued.

Edge cases.

• Refresh token expiry: after the refresh token itself expires, attempting to use it to get a new access token should fail cleanly, and the app should have a defined path (re-prompt the user to re-authorize) rather than an undefined error state.
• A user revokes consent after the fact, from their own account settings, for instance: on the next API call using a token issued under that consent, expect the token to be rejected, ideally immediately rather than only once the access token's own natural expiry is reached.
• Cross-origin flows: if the authorization or token exchange happens via a browser redirect across origins, verify the redirect URI is validated strictly (an app can't have its tokens redirected to an attacker-controlled URI by supplying an unregistered redirect target).

Compliance and security checks.

• The authorization code itself should be single-use and short-lived; confirm a second exchange attempt with the same code fails.
• State parameter validation, to confirm the flow is resistant to CSRF-style injection of a forged authorization response. CSRF (cross-site request forgery) here means an attacker tricking a logged-in user's browser into completing an authorization it never intended. The state parameter defeats this: the app generates a random value, sends it out with the authorization request, and then refuses to accept any callback whose returned state doesn't echo that exact value back, so an attacker who cannot see or guess that value cannot forge a callback the app will accept.

Automate vs. manual. Everything above except the very first step, the user actually clicking "approve" in a real browser, can be automated: the token exchange, error states, refresh flow, and revocation checks are all API-level behaviors testable without a human in the loop, using a test OAuth client and either a scripted browser or a test identity provider that can be driven programmatically. The literal human-consent click is the one piece best verified with a lightweight manual or scripted-browser smoke test periodically, not because it can't be automated at all, but because the ROI of fully automating a UI interaction that rarely changes is usually lower than automating the API-level behaviors around it.

Trade-offs and pitfalls

The edge cases (refresh token expiry, revocation, redirect URI validation) are the ones most often left out of an initial acceptance-criteria pass, because they only become relevant well after the "does login work" milestone has already shipped and felt done. Naming them explicitly in acceptance criteria up front, rather than discovering them as later bug reports, is the actual value of writing this list out before implementation starts, not after.

Direct answer: Combine three complementary layers, deterministic concurrency tests (a small number of hand-constructed, EXACT interleavings verified against a known-correct outcome) for precision, broad stress tests (many concurrent operations under real scheduling) for coverage, and a race-detector/model-checker tool for catching classes of bugs neither of the other two reliably finds, since no single technique alone gives both precision and coverage for concurrency correctness.

Structured elaboration

Deterministic concurrency tests: construct a SPECIFIC, exact interleaving (thread A does step 1, then thread B does step 1, then thread A does step 2,...) using explicit synchronization primitives (barriers, latches) to FORCE that exact order, and assert the outcome matches what's correct for that specific interleaving. Example, for a seat-reservation service: force two threads to both read "1 seat available" before either writes, confirming the system correctly rejects the second reservation attempt (via optimistic locking or an equivalent mechanism) rather than allowing a double-booking, a precise, deterministic test of the EXACT race scenario that matters most for this domain.

Stress tests: run many concurrent operations (hundreds or thousands of simulated reservation attempts across many threads) under REAL, uncontrolled scheduling, and verify an aggregate invariant holds (total successful reservations never exceeds total seat capacity), the same workload-generator-plus-verifier-oracle pattern covered in the concurrent-queue-harness sub-area, applied here to a domain-specific invariant.

Tools, race detectors: language/runtime-specific tools (Java's tooling for detecting unsynchronized shared-memory access, or Go's built-in -race flag) that instrument memory access and flag genuinely UNSYNCHRONIZED concurrent access to shared state, even if that specific access pattern didn't happen to produce an observably wrong RESULT on this particular run, catching latent races before they manifest as an actual, visible bug.

Tools, model checkers: systematically (often exhaustively, for a bounded state space) explore possible thread interleavings rather than relying on the OS scheduler to happen to hit the interesting ones; particularly valuable for a SMALL, critical piece of concurrency logic (the specific reservation-locking mechanism itself) where exhaustive interleaving coverage is both feasible and high-value, though generally too expensive to apply to an entire large system.

Injecting delays/failures or scheduling threads: for the deterministic tests, explicit synchronization (barriers/latches, as above) forces exact ordering; for stress tests, a chaos-injection layer (per the concurrent-queue-harness sub-area) deliberately widens race windows to make rare interleavings manifest more reliably without requiring full scheduling control.

Making concurrency failures reproducible for debugging: capture the FULL state at the moment of a detected failure (thread dump, the specific sequence of operations each thread had performed) so a stress-test failure, even though not perfectly re-playable via a seed alone (per the honest limitation covered in the concurrent-queue-harness sub-area), at least gives a detailed forensic starting point; for the highest-value cases, invest in converting a stress-test-discovered failure into a NEW deterministic test that forces the exact interleaving that caused it, once understood, giving a permanently reproducible regression test going forward.

Worked example: a seat-reservation service's deterministic test forces two threads to simultaneously attempt reserving the LAST available seat via an explicit barrier ensuring both threads pass their availability CHECK before either performs their WRITE, precisely reproducing the classic check-then-act race; the correct implementation (using SELECT... FOR UPDATE or optimistic locking, per the database-race-condition sub-area) should have exactly ONE of the two threads succeed and the other receive a clean "sold out" rejection, verified deterministically on every run of this specific test, not merely "usually" correct under uncontrolled stress-test timing.

Trade-offs & pitfalls: deterministic tests give precise, permanent regression coverage for the SPECIFIC scenarios you thought to construct, but by design cannot discover a race you didn't anticipate; stress tests and race-detector tools cover a much broader space but with correspondingly less precision about exactly what went wrong when they do find something; relying on only one of the three layers leaves a real gap, deterministic tests alone miss unanticipated races, stress tests alone are harder to debug precisely, and tooling alone (race detectors/model checkers) doesn't validate DOMAIN-SPECIFIC correctness (the seat-reservation invariant itself), only the absence of unsynchronized access.

For a RESTful web application (React SPA, Node.js REST API, PostgreSQL, external payment gateway), each pyramid tier should own a different, non-overlapping slice of confidence, and the tests below make that concrete.

Unit tier (four examples)

1. Discount/price calculator: a pure function calculateDiscount(price, tier); verifies core business math, e.g. Jest for the frontend or a plain test runner on the backend.
2. React component render logic: does the checkout form component render a validation error when the card field is empty; React Testing Library.
3. Request validator: does the order-creation handler reject a negative price before touching the database; a plain Jest unit test with mocked input, no HTTP or DB involved.
4. Payment-gateway response parser: given a sample JSON response from the gateway, does your parser extract the correct transaction ID and status; a pure-function Jest unit test with a hard-coded fixture, no real network call.

Each of these verifies one piece of logic in isolation and depends on nothing external, which is why they can run in milliseconds.

Integration tier (four examples)

1. API-to-database write path: POST an order to the real Node API running against a real (test) PostgreSQL instance, then query the database directly to confirm the row and its computed total are correct; Supertest plus a real Postgres test container.
2. Repository layer against Postgres: an ORM query (e.g., a Prisma or TypeORM query) that joins orders and customers, run against a seeded test database via Testcontainers, to catch a wrong join or a migration mismatch a mocked-DB unit test would miss.
3. Payment-gateway client against the gateway's sandbox: using an HTTP client library such as Axios (or the gateway's official Node.js SDK if one is provided), call the gateway's real sandbox endpoint (not your parser in isolation) to confirm your client sends a well-formed request and correctly handles the sandbox's real success and decline responses.
4. React SPA against a mocked API layer: render the checkout page and confirm it calls the real API client code (not the component logic alone) and correctly updates state on a real HTTP response, using a tool like MSW (Mock Service Worker) to intercept only the network boundary, not the application code.

Each of these proves two real components agree on a contract (route shape, SQL schema, gateway request format) that a unit test, by construction, cannot check because it never invokes the second component for real.

End-to-end tier (four examples)

1. Full checkout journey: drive the real React SPA in a real browser through add-to-cart, checkout form, and payment, against the real (or sandboxed) full stack, using Playwright or Cypress, to prove the whole assembled system delivers a working checkout.
2. Payment failure path end-to-end: using Playwright or Cypress, submit a card the gateway's sandbox is configured to decline, and confirm the SPA shows the correct user-facing error, proving the failure path is wired correctly all the way through, not just handled by the parser in isolation.
3. Session and auth flow: using Playwright or Cypress, log in, add an item, refresh the page, and confirm the cart persists, exercising the real session/cookie mechanism no lower-level test touches.
4. Order confirmation and receipt: using Playwright or Cypress to complete the purchase, combined with a test email-capture tool such as Mailhog or Mailtrap, confirm a confirmation email or receipt page reflects the correct final total, proving the pricing logic, the database write, and the presentation layer all agree once wired together for real.

Why each test belongs where it does

The dividing line is what would have to be REAL for the test to fail the way it's meant to: the unit tests fail only if the pure logic is wrong; the integration tests fail only if two real components disagree, even when each one's internal logic is correct in isolation; the end-to-end tests fail only if something in the full assembly, including things no lower test can see (routing, session state, real gateway behavior), is broken.

Trade-offs and pitfalls

The most common mistake with this stack specifically is testing the payment-gateway integration primarily at the end-to-end level because "it's the riskiest part": that inflates the slowest, flakiest tier with coverage that a much cheaper integration test against the gateway's sandbox could provide almost as well. Reserve end-to-end for the few journeys where the VALUE is specifically in proving the pieces are wired together, and push everything else down a tier.

Overview / approach
Describe risks per category, design deterministic tests that exercise serialization, file I/O, path handling, text encoding, newline normalization, and collation. Run tests on Linux, Windows, macOS in a CI matrix and on local VMs/devboxes. Automate platform-specific assertions rather than OS heuristics.

Concrete tests (what & why)

• Endianness in binary serialization
  • Test: serialize structured data (ints, floats, structs) on each platform using both host-endian and explicit little/big endian formats; verify round-trip and cross-platform compatibility.
  • Example: generate file on Windows (LE), read on Linux (LE), also read BE files and assert conversion.
• Filesystem case sensitivity
  • Test: create files whose names differ only by case (e.g., Readme.txt vs readme.txt); assert expected behavior (both exist on case-sensitive, conflict or overwrite on case-insensitive).
• Max path length
  • Test: create nested directories to approach platform max (Windows MAX_PATH, long-path prefix), try file open, copy, delete; assert graceful failure and clear error codes.
• Path separator & encoding differences
  • Test: construct paths using backslash/forward-slash; verify API normalizes correctly; include Unicode path components (e.g., emojis, accented chars).
• Newline conventions
  • Test: write files with LF, CRLF, CR; ensure parser normalizes and round-trips without data loss.
• Locale / collation variations
  • Test: sort/search strings containing locale-sensitive chars (ß, ñ, Turkish dotted/dotless i) under different locales; assert behavior matches expected collation tables or explicit locale settings.

Automation plan / CI

• Use platform matrix (GitHub Actions / Azure Pipelines) with runners for ubuntu-latest, windows-latest, macos-latest.
• Run tests in container/VM that can simulate edge cases (set LANG, LC_COLLATE, create files with special encodings).
• Include a small cross-platform test harness (Python/Node) that:
  • Generates corpus files (binary BE/LE, long paths, case-variants, newline variants, Unicode names)
  • Performs operations and asserts outcomes with clear failure messages
• Example GitHub Actions matrix snippet:

name: cross-platform-tests
on: [push, pull_request]
jobs:
  test:
    runs-on: ${{ matrix.os }}
    strategy:
      matrix:
        os: [ubuntu-latest, windows-latest, macos-latest]
    steps:
      - uses: actions/checkout@v4
      - name: Setup Python
        uses: actions/setup-python@v4
        with: python-version: '3.11'
      - name: Run cross-platform tests
        run: python tests/run_cross_platform_tests.py

CI best practices

• Use deterministic seeds and fixtures; commit test corpora.
• Fail fast with clear OS-tagged logs; collect artifacts (failed files, diffs).
• Run locale-sensitive tests under multiple locale env vars via matrix or dynamic runner configuration.
• Add nightly runs on real hardware VMs for exhaustive long-path/case tests.

Metrics & triage

• Track flaky tests by OS, add retries only for infra flakiness.
• Record failures with reproducer artifacts and suggested priority (data-loss, correctness, UX).

Direct answer

State the disagreement plainly and early rather than sitting on it, back it with the most objective evidence available, real numbers, a written spec, a shared definition, rather than opinion, and aim explicitly for a resolution: your manager changes their mind, you change yours, or you agree on a middle path, not just getting the disagreement heard and left unresolved.

Structured elaboration

Voice it constructively. Raise it privately and promptly, frame it as "here's what I'm seeing differently and why" rather than "I think you're wrong," and explicitly invite your manager to correct your understanding, since they may have context you do not.

Gather evidence before the conversation, not during it. Pull the most neutral data available beforehand, actual numbers, a written specification, a record of a prior decision, rather than relying on impressions in the moment. If the disagreement is really about a definition, what counts as "on time," what a metric actually measures, write both definitions down side by side so the gap is concrete instead of implied.

Present the evidence, not a verdict. Show the data and let your manager weigh it alongside context you might not have, like organizational priorities or input from other stakeholders you are not in the room for.

Aim for an actual resolution. Name the possible outcomes explicitly: your manager's original call stands with a documented reason, your alternative is adopted, or a genuine middle path is agreed, rather than letting the conversation end ambiguously with both sides still privately unconvinced.

Follow through visibly, including on the parts you did not get your way on. A disagreement voiced well but then quietly ignored afterward damages trust either way.

Worked example

As a Data Analyst, a manager gave feedback that a dashboard's "active user" metric should count anyone who opened the app, but the analyst believed that definition overstated real engagement and should instead require a meaningful action inside the app. Rather than silently implementing the manager's definition or quietly using a different one, raised the disagreement directly: pulled data showing what the numbers looked like under each definition side by side, including how the "open only" version made a recently launched feature's adoption look artificially strong. The manager had context the analyst did not, the "open only" definition already matched a metric reported externally to stakeholders, so changing it would break comparability with past reports. They agreed on a middle path: keep "open only" as the external metric for continuity, and add a second, internal-only metric using the stricter definition specifically for judging whether new features were actually being used.

Trade-offs and pitfalls

Bringing data does not guarantee winning the disagreement, and treating it as ammunition rather than shared information turns a discussion into a fight. Voicing disagreement late, after a decision has already shipped, forces a costlier reversal than raising it early would have. And accepting a "compromise" that is really capitulation dressed up as agreement does not resolve anything, it just delays the same disagreement to the next round.

Direct answer
For a sort-and-page service over millions of records, the edge cases split into two families: correctness of the ordering itself (duplicate sort keys, stable versus unstable sorting, null/absent keys, comparator exceptions) and correctness of the PAGINATION on top of that ordering (inconsistent ordering across pages, serialization differences, memory pressure at scale). The single most dangerous edge case, and the one a code review will not catch by inspection, is heavy duplication in the sort key combined with offset-based paging: without an explicit tiebreaker, the exact same dataset can silently drop or duplicate rows across a page walk, even though each individual page looks correct in isolation.

Structured elaboration

Unit-level (comparator/ordering) tests

• Duplicate sort keys: many rows sharing the same key value. Stability matters here: a STABLE sort preserves the original relative order of equal-key rows across repeated runs on the same input; an UNSTABLE sort (a plain quicksort, or many distributed-engine parallel sorts) does not guarantee this, and the client-visible symptom is that two consecutive page requests for the same query can return duplicated-key rows in a DIFFERENT relative order, which breaks the natural expectation that "page 2 continues exactly where page 1 left off."
• Null/absent keys: decide and test an explicit placement rule (nulls first or nulls last), since most SQL and sort libraries default to a specific, sometimes surprising, placement that differs across databases and languages.
• Comparator exceptions: a custom comparator that assumes a specific type (e.g. always numeric) will raise on a heterogeneous or malformed value; test that the exception surfaces as a clear, typed validation error at ingestion time, not a raw crash deep inside the sort call during a customer-facing request.

Integration-level (pagination-consistency) tests

• Inconsistent ordering across pages: walk ALL pages of a query, concatenate every returned ID, and assert the concatenated set exactly equals the known full ID set for that dataset (no duplicates, no omissions), rather than eyeballing individual pages.
• Serialization differences: if the API paginates via an opaque cursor token, confirm the cursor's meaning is stable across a service redeploy (a cursor encoding a raw row offset breaks if the underlying storage is repartitioned; a cursor encoding a stable sort-key value plus a tiebreaker id survives it).
• Memory pressure: for millions of records, assert the service streams/paginates from the underlying store rather than materializing the full sorted result in memory before paging; a targeted test can seed a dataset sized to exceed a deliberately small memory limit in a test environment and confirm the service still completes without an out-of-memory failure, rather than asserting a specific memory number.

End-to-end tests

• A full client-simulated walk (as in the worked example below) over a realistic dataset shape, verifying the concatenated, deduplicated result matches an independent full-sort oracle.

Data generation for large and pathological datasets

• Generate at multiple scales (thousand, million-row smoke tests) with a FIXED seed for reproducibility.
• Deliberately skew the generator toward pathological shapes: a large fraction of rows sharing one sort-key value (the duplicate-key stress case), a long tail of unique keys, explicit NULL injection at a known rate, and adversarial comparator inputs (mixed types if the schema technically allows it).

Worked example (executed): duplicate-sort-key pagination bug, caught and fixed

import random
random.seed(42)
rows = [{"id": i, "score": i % 4} for i in range(500)]  # 500 rows, only 4 distinct sort-key values

def unstable_source_page(all_rows, offset, limit, rng):
    grouped = {}
    for r in all_rows: grouped.setdefault(r["score"], []).append(r)
    ordered = []
    for k in sorted(grouped):
        bucket = grouped[k][:]; rng.shuffle(bucket)   # ties reordered on every call, simulating a real DB with no tiebreaker
        ordered.extend(bucket)
    return ordered[offset: offset + limit]

def stable_source_page(all_rows, offset, limit):
    ordered = sorted(all_rows, key=lambda r: (r["score"], r["id"]))   # explicit tiebreaker
    return ordered[offset: offset + limit]

def walk_all(page_fn, total, limit):
    seen = []
    for offset in range(0, total, limit):
        seen.extend(r["id"] for r in page_fn(offset, limit))
    return seen

LIMIT = 37
rng = random.Random(7)
unstable_ids = walk_all(lambda o, l: unstable_source_page(rows, o, l, rng), 500, LIMIT)
stable_ids = walk_all(lambda o, l: stable_source_page(rows, o, l), 500, LIMIT)

expected = set(r["id"] for r in rows)
print(len(unstable_ids), len(set(unstable_ids)), len(unstable_ids) - len(set(unstable_ids)), len(expected - set(unstable_ids)))
print(len(stable_ids), len(set(stable_ids)), len(stable_ids) - len(set(stable_ids)), len(expected - set(stable_ids)))

Executed output: the unstable (no-tiebreaker) walk fetched 500 rows total but only 353 unique ids, 147 duplicated across page boundaries and 147 missing entirely from the walk, on the exact same 500-row dataset. The tiebreaker-stabilized walk fetched 500 rows, 500 unique, 0 duplicates, 0 missing. This is not a contrived pathological case, it is what happens on ANY dataset where the sort key alone does not uniquely order the rows and pagination crosses a tie boundary.

Trade-offs & pitfalls
The most common mistake is trusting that "the database sorts consistently" without an explicit tiebreaker column; many query engines make NO ordering guarantee for tied rows across separate query executions, especially under parallel execution plans, so relying on incidental stability is a latent bug that a small-scale manual test will not surface (the bug only shows up once ties outnumber a single page, which small test datasets rarely do). A second pitfall is testing pagination correctness only at the unit level (one page looks right) without the end-to-end concatenate-and-diff check across the FULL walk, which is the only way this class of bug is actually visible. Third, testing memory pressure by asserting a fixed byte count is fragile across environments; asserting that the operation SUCCEEDS under an artificially constrained limit, and that the service degrades to a documented, bounded failure mode (not an unbounded materialization) rather than crashing, is the more durable test design.