Microsoft QA Engineer Interview Preparation Guide (Mid-Level)

Direct answer: Build the business case around a concrete cost the stakeholders already care about (blocked releases, wasted engineer-hours, or a near-miss incident), quantify it honestly with real numbers you can defend, and negotiate a SCOPED commitment (a specific number of tests, a specific timeframe) rather than an open-ended ask, since a bounded proposal is easier for stakeholders to say yes to and easier for you to be held accountable to delivering.

Structured elaboration

• Building the business case with metrics: pull concrete numbers directly relevant to what the stakeholder audience cares about, engineering leadership responds to developer-velocity metrics (hours lost per week to flaky-test investigation, average PR cycle time inflated by reruns); product/release stakeholders respond to release-cadence metrics (how many releases were delayed or how much manual verification time was spent working around untrusted CI). Use REAL numbers pulled from CI history and telemetry, not estimates presented as measured facts, since a stakeholder who later discovers a cited number was invented loses trust in everything else in the proposal.
• Demos: a live or recorded demonstration of the actual PAIN, showing a specific PR blocked by a test everyone knows is unrelated to the change, or walking through the actual triage time spent on a recent incident, is often more persuasive than an abstract metric, because it makes the cost concrete and relatable rather than an aggregate statistic stakeholders have to trust at face value.
• Risk analysis: connect flakiness to a SPECIFIC risk stakeholders already track, for example, "a flaky test being routinely dismissed increases the chance a real regression ships, and here's a near-miss where that almost happened" is a materially different, more urgent argument than "our test suite has some flaky tests."
• Negotiating scope and timelines: propose something SPECIFIC and time-boxed (for example, "dedicate 20% of one team's capacity for one quarter to fix the top 20 highest-impact flaky tests, based on this prioritization") rather than an open-ended "let's fix flakiness," which is much harder for a stakeholder to commit resources against and much harder for you to be held accountable to.
• Outcomes to report back: both technical (a measured before/after flakiness-rate or blocked-merge-time reduction) and process (a resulting ownership model, an SLA that persists after the initial investment), since a purely technical win that reverts once attention moves elsewhere is a weaker long-term outcome than one that also changed how the organization operates going forward.

Worked example (a realistic hypothetical, per the question's own fallback framing): I was asked why release cadence had slipped two cycles in a row. Rather than answering with impressions, I pulled CI history showing 23% of blocked-merge time over the prior month traced to just 6 tests, and demonstrated one specific incident where a real regression's failure was initially dismissed as "probably just flaky" because the same test had a history of noisy failures, costing a day of delayed detection. I proposed a scoped ask: 2 engineers, 3 weeks, fix those specific 6 tests, with a follow-up ownership process to prevent regression. Leadership approved the scoped ask (much smaller and easier to say yes to than "invest in test reliability generally"), and afterward blocked-merge time from those 6 tests dropped close to zero, with the ownership process persisting as the durable process outcome; that concrete before/after became the reference case for a subsequent, larger investment the following quarter.

Trade-offs & pitfalls: leading with a scary-sounding but poorly-substantiated number ("flaky tests are costing us millions") without a defensible derivation invites skepticism and can set back the case rather than helping it; better to under-claim with real, defensible numbers than over-claim with invented ones. A second pitfall: negotiating for MORE scope than you can realistically deliver in the agreed timeline undermines the case for the NEXT ask, since a missed commitment is remembered longer than a modest, delivered one.

Direct answer. The Page Object Model (POM) is a design pattern that wraps each page (or reusable UI region) in a class exposing locators and high-level actions, so tests call methods like login_page.login(user, pw) instead of touching selectors directly. It separates what a test wants to do from how the page currently implements it, so a UI change means editing one class, not every test that touches that page.

Structured elaboration.

• Responsibilities of a page object: hold the locators for that page/component, expose action methods (login, add_to_cart) and query methods (is_logged_in, get_price) that return plain data, never raw WebElement handles, so assertions stay in the test.
• Pros: one place to fix a broken locator; tests read like user actions instead of DOM manipulation; onboarding is faster because a new page's shape is discoverable from its class.
• Cons: a naive implementation adds an extra layer of indirection for very small suites; if page objects grow "god classes" covering unrelated flows, they become as brittle as the tests they replaced.
• Anti-patterns to avoid:
  1. Assertions inside the page object ("logic in POM") - a page object that calls assert couples it to one test's expectations and can't be reused by a different test that wants a different check.
  2. Locators or raw waits inside the test ("logic in tests") - defeats the whole point; the test should never see a CSS selector.
  3. Missing explicit waits - a page object that clicks immediately after navigation, with no wait for the target element, is the single most common source of flaky POM-based suites.
• Structuring page/component objects: for anything reused across pages (a nav bar, a product card, a modal), extract it as its own component-object class that a page composes by reference, rather than duplicating its locators/actions on every page that contains it. Locators, actions, and assertions get their own layer: locators are private constants, actions are public methods that use them, assertions live only in the test (or in an assertion helper the test calls).

Worked example. A minimal Python layout for a small login-flow project:

tests/
  test_login.py          # calls LoginPage methods, asserts on plain return values
pages/
  login_page.py          # class LoginPage: locators + navigate()/login()/error_message()
  base_page.py            # shared wait helpers all page objects inherit

class LoginPage(BasePage):
    USERNAME = ("id", "username")
    PASSWORD = ("id", "password")
    SUBMIT = ("id", "submit")
    ERROR = ("css selector", ".error-banner")

    def login(self, username, password):
        self.type_into(self.USERNAME, username)
        self.type_into(self.PASSWORD, password)
        self.click(self.SUBMIT)
        self.wait_until_visible(self.ERROR, optional=True)

    def error_message(self):
        return self.text_of(self.ERROR) if self.is_visible(self.ERROR) else None

test_login.py calls LoginPage(driver).login(...) and asserts on error_message() - it never sees a locator.

Trade-offs and pitfalls. POM is not free: over-abstracting a page object into a "framework within the framework" (generic perform(action_name, **kwargs) dispatchers) trades locator duplication for indirection nobody can trace during a failure. The senior judgment call is knowing when a shared component genuinely appears on multiple pages (extract it) versus when two pages only look similar today and will diverge (don't prematurely extract, or the shared class becomes a tangle of if page_type == ... branches).

Direct answer
Case-insensitive email dedup looks like a one-line LOWER() fix, but the test data has to cover six distinct failure modes: exact duplicates, NULL, empty strings, casing differences, surrounding whitespace, and Unicode-normalized forms that look identical but are different byte sequences. The last one is the trap: LOWER(TRIM(email)) handles the first five correctly but does NOT dedupe Unicode-normalization variants, and that has to be surfaced as a real limitation, not silently assumed away.

Structured elaboration
Each row category exercises a different part of the normalization logic:

• Duplicates and casing: Alice@Example.com, alice@example.com should collapse to one identity; this is the core equivalence class the feature exists for.
• Leading/trailing whitespace: " alice@example.com " is semantically the same address but a byte-different string; a dedup that only lowercases without trimming will keep it as a false-distinct row.
• NULL and empty string: these are two DIFFERENT edge cases, not one. NULL fails equality comparisons in SQL (NULL = NULL is NULL, not TRUE), so a naive GROUP BY email silently puts every NULL row in its own untouched bucket (or drops them, engine-dependent) rather than merging them, while an empty string '' is a valid (if useless) value that DOES group correctly with other empty strings. The dedup logic must decide, explicitly, whether NULL and '' count as "no email" (excluded from dedup entirely) or as data to be deduplicated like any other value.
• Unicode-normalized forms: café@example.com can be encoded as one composed codepoint for é (Normalization Form Canonical Composition, NFC) or as e followed by a separate combining acute accent character (Normalization Form Canonical Decomposition, NFD). Both render identically on screen and are the same logical string, but they are different byte sequences, so LOWER()/TRIM() treat them as distinct.

Worked example
Test dataset (SQLite, run for real):

CREATE TABLE users (id INTEGER PRIMARY KEY, email TEXT);
INSERT INTO users (id, email) VALUES
  (1, 'Alice@Example.com'),
  (2, 'alice@example.com'),
  (3, '  alice@example.com  '),
  (4, 'BOB@EXAMPLE.COM'),
  (5, NULL),
  (6, ''),
  (7, 'carol@example.com'),
  (8, 'café@example.com'),                       -- NFC: e-with-acute is one codepoint
  (9, 'cafe' || X'0301' || '@example.com');       -- NFD: plain e + combining acute accent (U+0301)

SELECT LOWER(TRIM(email)) AS normalized_email, COUNT(*) AS row_count, GROUP_CONCAT(id) AS ids
FROM users
WHERE email IS NOT NULL AND TRIM(email) != ''
GROUP BY LOWER(TRIM(email))
ORDER BY normalized_email;

Actual output:

This confirms the expected result for the first five categories (rows 1, 2, 3 correctly collapse to one alice@example.com group of 3; BOB@EXAMPLE.COM and carol@example.com are correctly left alone; NULL and '' are correctly excluded by the WHERE clause rather than silently grouped together) and PROVES the Unicode gap: rows 8 and 9 are the same visual email address but land in two separate groups, because LOWER(TRIM()) operates on bytes, not on normalized Unicode form. Fixing this requires normalizing to NFC (or NFD, consistently) at the application layer before the dedup query runs, since standard SQLite has no built-in Unicode normalization function.

Automating setup and teardown: seed the fixture rows above from a versioned SQL or JSON fixture file (not ad hoc INSERTs typed in each test), so the case list is reviewed and diffed like code. Each test should run inside its own transaction: BEGIN, insert fixtures, run the dedup query, assert, then ROLLBACK (or, for engines/frameworks without cheap savepoints, TRUNCATE and re-seed between tests). This keeps the 9-row dataset above isolated per test run so tests can execute in parallel and in any order without one test's leftover rows corrupting another's expected row_count.

Trade-offs and pitfalls
The most common pitfall is writing the dedup query, seeing it pass on a small hand-built dataset, and never testing a genuine Unicode-normalization collision, because NFC and NFD variants of the same string look byte-identical when eyeballed in a terminal or spreadsheet. A second pitfall is treating NULL and '' as the same case: they require different WHERE/COALESCE handling and different product decisions (is a blank email a data-quality bug to flag, or a legitimately unset optional field to ignore?). Finally, resist "fixing" the Unicode gap by adding a COLLATE NOCASE-style trick in SQL: collations control comparison behavior, not byte-level Unicode normalization, so that only strengthens the casing case, not the NFC/NFD case; the real fix is normalizing at the application or ETL layer with a Unicode-aware library before the row ever reaches this query.

Direct answer

A typical CI/CD pipeline moves a change through five kinds of work: verify the code compiles and passes fast checks, verify it behaves correctly in isolation, verify it behaves correctly with its dependencies, package it into something deployable, and move that package safely into production. Concretely: checkout, build, static analysis, unit tests, integration tests, artifact publish, deploy, smoke test. Which of these run on every pull request versus only on merge to main is a deliberate trade-off between fast feedback and thoroughness.

Structured elaboration

Checkout and build. Pulls the commit, resolves dependencies, and compiles or bundles the code. This always runs on every PR and every merge; if it fails, nothing downstream is worth running. Budget: seconds to a couple of minutes for most services.

Static analysis (lint, type-check, and any fast security linting). Cheap and deterministic, so it runs on every PR alongside the build. It catches an entire class of bugs (unused variables, obvious type errors, banned patterns) before a human or a slower test even looks at the change.

Unit tests. Exercise a function or module in isolation, with dependencies mocked or stubbed. These run on every PR because they're fast (seconds to low minutes for a healthy suite) and directly test the code the author just wrote.

Integration tests. Exercise the service against real or near-real dependencies (a real database, a real message queue, or a called service). These are slower and flakier than unit tests, so many teams run a fast subset on every PR and the full suite on merge to main or on a schedule.

Artifact publish. Package the build output (a container image, a JAR, a wheel) and push it to a registry with an immutable identifier. This typically only happens on merge to main or on a tag, not on every PR, because you don't want to publish a candidate for every work-in-progress commit.

Deploy and smoke test. Deploy the published artifact to an environment and run a small number of fast checks against the live service (does it start, does the health endpoint return 200, can it serve one representative request) before declaring the deploy successful. This runs after publish, gated by whatever approval policy the target environment requires.

Deciding the runtime budget per stage. The real design constraint is total pipeline latency on the PR path, because that's what blocks a developer. A common target is keeping the PR-blocking stages (build, lint, unit tests, and a fast integration-test subset) under 10 minutes combined, and pushing anything slower (full integration suite, load tests, security scans that take longer) to run on merge or nightly instead of on every PR. If a stage regularly exceeds its budget, that's a signal to parallelize it, cache more aggressively, or move it later in the pipeline rather than let it silently erode developer feedback speed.

Worked example

A small service's pipeline might budget: checkout+build 90s, lint+unit tests 60s (run in parallel with build where the toolchain allows), a fast integration-test subset (only tests touching changed files) 3 minutes, giving a PR-blocking total of roughly 5 minutes. On merge to main, add: full integration suite 12 minutes, artifact publish 1 minute, deploy to staging 2 minutes, smoke tests 30s. The PR path optimizes for developer feedback speed; the merge path optimizes for release confidence, and it's acceptable for it to take longer because it doesn't block anyone's next commit.

Trade-offs and pitfalls

The most common mistake is running the full test suite (including slow integration and end-to-end tests) on every PR: it maximizes confidence per commit but destroys feedback speed, and teams end up merging on red or batching PRs to avoid the wait, which defeats the purpose of continuous integration. The opposite mistake, running almost nothing on PR and deferring everything to merge, means breakages are discovered after they've already landed on main, which is more expensive to fix than catching them before merge. The healthy middle ground is a small, fast, high-signal PR gate and a slower, more thorough merge/nightly gate, with the two suites kept in sync so a PR-passing change doesn't routinely fail on merge for reasons the PR gate could have caught cheaply.

Situation & goal
Design a library that lets QA reliably locate and wait for elements rendered in canvas/WebGL (no DOM), supporting pixel, hit-test, OCR, and testing hooks, and integrate with Playwright/Selenium.

APIs — Locator surface

• Locator.pixelMatch(selector, region?, options)
  • Finds positions by template image or pattern; options: threshold, tolerance, scale.
• Locator.hitTest(x,y) / Locator.hitTestById(renderId)
  • Uses injected runtime hit-test API (see hooks).
• Locator.ocr(region, lang?, options)
  • Returns text boxes and confidence via Tesseract or bundled OCR.
• Locator.hook(name, predicate?)
  • Locates items exposed by in-app testing hooks (renderId, metadata).

Example usage:

const loc = await CanvasLoc.create(page);
const pos = await loc.pixelMatch('sprite.png', {region:{x:0,y:0,w:800,h:600}, threshold:0.9});

Synchronization APIs

• Sync.waitForFrame(fn?, timeout)
  • Wait until next n animation frames or predicate based on readback.
• Sync.waitForRenderState({pixelsChanged:<n>, noChangeMs:<ms>, hookStatePredicate, ocrContains})
  • Combines strategies (frame count + visual diff + hook signals).
• Sync.pollWithBackoff(checkFn, {intervals})
  • Resilient polling using exponential backoff calibrated to 60fps.

In-app testing hooks (required for stability)

• Expose a small testing channel on window.TESTING
  • registerRenderable(id, metadata), hitTest(x,y)->id, getState(id) -> serializable state
• Library injects a tiny shim into the page to call these hooks over evaluate().

Implementation notes

• Pixel matching uses WebGL readPixels or canvas.toBlob -> compare via SSIM/normalized cross-correlation; prefer GPU readback when available.
• OCR: run off-main-thread via Web Worker using Tesseract or lightweight ML model.
• Hit-test: best if app exposes GPU-side IDs; otherwise implement color-encoded hit buffer render pass.

Playwright/Selenium integration

• Playwright: bundle CanvasLoc as a helper; use page.exposeBinding for IPC and page.evaluate to inject shim. Provide async helpers that return Playwright Locator-like objects.
• Selenium: use executeScript / executeAsyncScript to install shim and exchange data (use base64 images for pixel readback).
• Provide adapters: PlaywrightAdapter {waitForFrame, pixelMatch} and SeleniumAdapter with identical API.

Reliability & trade-offs

• Prefer testing hooks + hit-test for determinism; fall back to pixel/OCR with thresholds.
• Minimize full-frame readbacks; use region diffs and hashed tiles.
• Add telemetry: capture failing diffs, thumbnails, and hook-state snapshots for debugging.

Example test pattern

1. Inject hooks in dev build.
2. Trigger action (click at locator.hitTest(...)).
3. Sync.waitForRenderState({hookStatePredicate: s => s.progress === 'done', timeout:5000}).
4. Assert via ocr/pixelMatch and capture evidence on failure.

This design balances determinism (hooks/hit-test) with fallback visual methods (pixel/OCR), and provides concrete adapters for Playwright and Selenium to make canvas/WebGL UIs testable and stable.

Direct answer

I acknowledge the loss briefly, then move quickly to figuring out what's actually salvageable and what the new priority needs, rather than dwelling on the work that no longer matters. I also close the loop with anyone who was expecting the original outcome, so they're not left assuming it's still coming.

Structured elaboration

• Triage what's salvageable fast. Most pivots leave more usable than it feels like at first: partial artifacts, research findings, or skills built along the way often carry over even when the original plan doesn't.
• Repurpose the salvage into the new direction on purpose, rather than discarding it out of frustration just because the original goal changed.
• Communicate the change to anyone expecting the original outcome, plainly and as soon as reasonable, rather than letting them find out later or assume things are still on track.
• Look afterward for what made the work exposed to being wasted in the first place, such as working in a large chunk before checking in, or not surfacing the risk of change earlier, and adjust that, even with a small process tweak, so less is exposed to the same risk next time.
• The same shape applies if what got displaced is a personal learning plan rather than a project: the actual skill or knowledge gained usually still carries over even if the plan itself gets scrapped.

Worked example

Partway through a quarter, our team's roadmap shifted after a strategy change, and a chunk of research and early build work I'd put real effort into stopped being relevant. I spent a short amount of time being honestly annoyed about it, then turned to what was salvageable: the research into user behavior I'd done for the shelved feature turned out to apply almost directly to the new priority, since it was really about understanding the same users, just answering a different question. I reused that research rather than starting fresh, which saved a real amount of time on the new work. I also reached out directly to a couple of stakeholders who'd been expecting the original feature, to let them know the change and why, rather than letting them discover it when it quietly disappeared from a roadmap update. Afterward, I mentioned in a retro that we'd been working in one large chunk without checking in with the wider team, which was part of why the change hit so late and wasted more than it needed to; we started doing shorter check-ins on longer efforts after that.

Trade-offs and pitfalls

The clearest trap is visible frustration or dwelling on the sunk work, which mostly just reads as inflexibility rather than helping anything. A subtler one is not actually looking for what's salvageable, and treating the whole effort as wasted out of frustration when a decent chunk of it usually still applies. The other common miss is not communicating the change to the people who were expecting the original outcome, which just moves the surprise downstream to them instead.

Direct answer

Detecting the conflict is the easy part: run every published contract's interactions against the provider and look for contradictory expectations on the same request, one consumer expecting field A present, another expecting it absent, for example. Resolving it is the harder part, and it's fundamentally a negotiation between teams that tooling can only support, not replace.

Structured elaboration

Detection. Because every consumer's contract is verified independently against the same provider code, a genuine conflict shows up as two contracts making incompatible assertions about the same request/response pair. Comparing contracts pairwise, or having the broker (the shared contract-testing service, such as a Pact Broker, that every consumer publishes its contract to and every provider's build verifies against) flag interactions with overlapping requests but divergent expected responses, surfaces this automatically rather than waiting for it to show up as a production bug for one side or the other.

Version negotiation. Once a conflict is found, the technical fix is usually to stop treating "the provider's response" as one fixed shape and instead let it vary by what the caller actually needs: a version header or content-negotiation scheme, so consumer A gets the shape it expects and consumer B gets a different, equally valid shape, from the same underlying data. This works well when the conflict is really "two valid interpretations of the same data" rather than one side simply being wrong.

Backward and forward compatibility. The provider's own compatibility policy determines how much freedom there is here: if it commits to strict backward compatibility, existing consumer expectations become close to a hard constraint on any change, and new consumer needs have to be satisfied additively. If it's willing to introduce controlled breaking changes on a schedule, the negotiation instead becomes "which consumers need to migrate, and by when": publish the new behavior alongside the old one, give every affected consumer an explicit deadline, track which consumers have actually migrated (not just whether the deadline has passed), and only retire the old behavior once real usage has dropped to zero.

Preventing breakage during resolution. Whatever the resolution turns out to be, it needs to ship without breaking either consumer mid-negotiation. In practice that means introducing the resolved behavior as an addition, a new field, a new version, alongside the existing ones, verifying all affected contracts against it, and only removing the old behavior once every consumer that depended on it has migrated and its contract has been updated to prove that.

Organizational side. Technically resolving the conflict doesn't resolve WHY it happened. A conflict is often a sign that two consumer teams have differing, undocumented assumptions about what the provider is supposed to guarantee. The organizational fix is making the provider's actual contract, the intersection of what it's willing to promise, visible and owned, so the next new consumer integrates against a documented reality instead of guessing and creating the next conflict.

Trade-offs and pitfalls

Silently picking a winner, quietly changing the provider to satisfy whichever contract verified most recently, and letting the other consumer's contract simply start failing, is a bad outcome even though it looks like progress in CI (one team's test starts passing). It just moves the conflict onto the consumer whose contract now fails, with no one deciding that trade-off deliberately. The discipline that prevents this is treating any newly-conflicting contract as a signal to open a conversation between the teams involved, not as a bug in one contract to be silenced.

Direct answer
Code review becomes a coaching tool the moment you separate what has to change before this merges from what's worth teaching, and handle each differently, since blocking mixes poorly with explaining. What counts as the important risk to teach toward also shifts by what's being reviewed: correctness and style for typical application code, reproducibility and data leakage for ML work, and blast radius for infrastructure changes.

Separate blocking feedback from teaching feedback

• Mark comments explicitly as blocking versus non-blocking (or use a similar convention), so the author isn't left guessing what actually has to change before merge. Teaching comments that aren't required for merge belong in the non-blocking bucket, otherwise you either water down real teaching moments to keep the change unblocked, or block a mergeable change to make a point.
• Ask before you tell: a comment phrased as a question ("what happens if this list is empty?") invites the author to find the issue themselves, which teaches the underlying reasoning; a comment phrased as an instruction just transmits the fix.

What "the important risk" means shifts by artifact type

• Typical application code: the coaching focus is usually correctness, readability, and test coverage; the failure mode being taught against is a defect shipping or the next person not being able to follow the change.
• ML notebooks and experiment configs: the review risk is different in kind, not just degree. The critical things to check and teach toward are reproducibility (is the seed pinned, is the environment specified, can someone else get the same result) and data leakage (does the training data have any path back to the evaluation set, directly or through a shared preprocessing step). A notebook can be clean, readable code and still be dangerously wrong for reasons that have nothing to do with code style.
• Terraform and other infrastructure-as-code changes: the review risk is blast radius, not defects in the traditional sense. A small, correct-looking diff can still be catastrophic if it touches a shared resource or removes a safeguard. Coaching here means teaching someone to ask what does this affect beyond what's in the diff before asking is this line correct.

Making it a genuine teaching moment, not just a gate

• When there's something worth teaching, don't just fix it in the comment; explain the why, and where useful, point to a real example elsewhere in the codebase rather than a generic principle.
• For anything too deep to unpack asynchronously in a comment thread, offer a short pairing session instead of a long comment chain; some things teach faster live than in writing.
• Close the loop: after a pattern comes up more than once for the same person, raise it directly in a 1:1 rather than only ever surfacing it inside individual review threads, so it becomes a recognized growth area instead of a recurring surprise.

Worked example
Reviewing a teammate's change that added a new model training script, the code itself was clean and well-tested in the conventional sense. The actual coaching moment was elsewhere: the evaluation split was built after a preprocessing step that had already seen the full dataset, which meant the reported accuracy was optimistic in a way unit tests would never catch. Rather than just fixing the split order and moving on, the comment walked through why that ordering matters (what leakage actually does to the reported number) and pointed to another script in the repo where the split happened correctly, before the shared preprocessing step. That change did get blocked, since the leakage was a real correctness issue, but the teaching part was the explanation of why, not the fact that it was blocked.

Trade-offs and pitfalls

• Making every comment a teaching moment, including on merge-blocking issues, slows delivery and can read as review turning into a lecture; save the deeper explanations for the genuinely worthwhile ones and keep routine fixes routine.
• Applying the same review lens (say, defect-finding) to every artifact type misses the risks that matter most for that artifact; a Terraform change reviewed like application code will pass style and correctness checks while missing blast radius entirely.
• If teaching moments only ever show up as isolated review comments and never get named directly to the person as a pattern, growth stays implicit and slower than it needs to be.

Direct answer

Keep both. Store every attempt as an append-only raw record (the source of truth for ad-hoc investigation and for recomputing anything later), and maintain small pre-aggregated rollup tables for the dashboards (a rollup is a summary table with one row per day and suite holding counts, instead of one row per test attempt). Raw rows alone make dashboards slow and costly; rollups alone cannot answer a question you did not plan for, such as "which tests failed on this commit only on retry" (a flake, meaning a test that passes and fails on the same code, leaves exactly that trace).

Core versus optional. The core of the answer is two choices: keep raw rows plus a small rollup, and partition the raw table by day. Everything else below (the artifact pointer, the warehouse variant) is refinement.

Plain-language terms. Append-only means rows are only ever added, never edited. Partitioned by day means the database stores each day's rows in its own physical chunk. Idempotent key means a unique key that makes re-sending the same result harmless, because the duplicate is rejected instead of counted twice. Object storage is cheap file storage (such as S3) for big blobs, and a URI is just the address of a file there. A columnar warehouse (for example BigQuery) stores data column by column, which is fast for scanning huge history but is a later step. Cluster by means sorting stored data by chosen columns so related rows sit together.

Layout

• Raw table, one row per test attempt, partitioned by day on the start time. Dashboard and investigation queries nearly always filter by time, so the database reads only the relevant days, and retention becomes dropping an old partition instead of deleting millions of rows one by one.
• Index on (test id, start time descending) for per-test history queries.
• Idempotent key (run, test, attempt, start time) so a re-sent result cannot double-count. In PostgreSQL the primary key of a partitioned table must include the partition column.
• Large artifacts stay out. Keep logs and screenshots in object storage and store only a URI pointer in the row.
• Rollup table per day and suite with counts by status, rebuilt by a job that recomputes the last few days, so late-arriving data and re-runs are safe.
• In a columnar warehouse the same idea becomes partition by day and cluster by suite and test id.

Worked example (PostgreSQL, run unchanged; illustrative rows)

SET TIME ZONE 'UTC';
CREATE TABLE test_attempts (
  run_id        bigint      NOT NULL,
  test_id       text        NOT NULL,
  attempt       smallint    NOT NULL,
  suite         text        NOT NULL,
  team          text        NOT NULL,
  branch        text        NOT NULL,
  commit_sha    text        NOT NULL,
  status        text        NOT NULL,   -- passed | failed | skipped | quarantined
  started_at    timestamptz NOT NULL,
  duration_ms   integer     NOT NULL,
  error_sig     text,                   -- hash of the normalised error, NULL if passed
  artifact_uri  text,                   -- pointer to logs/screenshots, never the bytes
  PRIMARY KEY (run_id, test_id, attempt, started_at)
) PARTITION BY RANGE (started_at);

CREATE TABLE test_attempts_2026_09_28 PARTITION OF test_attempts
  FOR VALUES FROM ('2026-09-28') TO ('2026-09-29');
CREATE TABLE test_attempts_2026_09_29 PARTITION OF test_attempts
  FOR VALUES FROM ('2026-09-29') TO ('2026-09-30');

CREATE INDEX ON test_attempts (test_id, started_at DESC);

CREATE TABLE suite_daily (
  day date NOT NULL, suite text NOT NULL, team text NOT NULL,
  passed int NOT NULL, failed int NOT NULL,
  skipped int NOT NULL, quarantined int NOT NULL,
  PRIMARY KEY (day, suite)
);

INSERT INTO test_attempts VALUES
 (1,'t1',1,'checkout','pay','main','a1','passed', '2026-09-28 10:00+00',  900, NULL, NULL),
 (1,'t2',1,'checkout','pay','main','a1','failed', '2026-09-28 10:00+00', 1200, 'sig9', 's3://logs/1/t2'),
 (1,'t2',2,'checkout','pay','main','a1','passed', '2026-09-28 10:05+00', 1100, NULL, NULL),
 (2,'t1',1,'checkout','pay','main','b2','passed', '2026-09-29 10:00+00',  950, NULL, NULL),
 (2,'t2',1,'checkout','pay','main','b2','skipped','2026-09-29 10:00+00',    0, NULL, NULL);

-- Idempotent rollup: recompute a day from raw rows, safe to re-run.
-- GROUP BY 1, 2, 3 groups by the 1st, 2nd and 3rd selected columns (day, suite, team).
-- ON CONFLICT ... DO UPDATE means: if a row for that (day, suite) already exists,
-- overwrite it with the freshly computed values (EXCLUDED holds the new values).
INSERT INTO suite_daily
SELECT started_at::date, suite, team,
       count(*) FILTER (WHERE status = 'passed'),
       count(*) FILTER (WHERE status = 'failed'),
       count(*) FILTER (WHERE status = 'skipped'),
       count(*) FILTER (WHERE status = 'quarantined')
FROM test_attempts
WHERE started_at >= '2026-09-28' AND started_at < '2026-09-30'
GROUP BY 1, 2, 3
ON CONFLICT (day, suite) DO UPDATE
  SET passed = EXCLUDED.passed, failed = EXCLUDED.failed,
      skipped = EXCLUDED.skipped, quarantined = EXCLUDED.quarantined;

-- Dashboard trend: read only the small rollup.
SELECT day, suite, passed, failed FROM suite_daily ORDER BY day;

-- Per-test history: one test across days, served by the (test_id, started_at) index.
SELECT started_at, attempt, status FROM test_attempts
WHERE test_id = 't2' AND started_at >= '2026-09-28' ORDER BY started_at DESC;

-- Retention: drop a whole day, no row-by-row DELETE.
DROP TABLE test_attempts_2026_09_28;
SELECT count(*) AS rows_left FROM test_attempts;

    day     |  suite   | passed | failed 
------------+----------+--------+--------
 2026-09-28 | checkout |      2 |      1
 2026-09-29 | checkout |      1 |      0
(2 rows)

       started_at       | attempt | status  
------------------------+---------+---------
 2026-09-29 10:00:00+00 |       1 | skipped
 2026-09-28 10:05:00+00 |       2 | passed
 2026-09-28 10:00:00+00 |       1 | failed
(3 rows)

 rows_left 
-----------
         2
(1 row)

The rollup shows checkout with 2 passed and 1 failed on the 28th because t2 failed on attempt 1 and passed on attempt 2: the rollup counts attempts, which keeps the flake signal. If you want final outcomes instead, roll up only each test's last attempt. Say which one the dashboard means. Dropping the 28th's partition removed its rows in one statement, leaving the 2 rows from the 29th.

Sizing intuition. If 20,000 runs a day each hold 500 results, that is 10,000,000 raw rows a day (20,000 x 500), while a rollup by day and suite is one row per suite per day, so a dashboard reads thousands of rows, not millions. At an assumed 200 bytes a row (an estimate, not a measurement) the raw data is about 2 GB a day before compression, so a 90-day window is roughly 180 GB, which is why dropping old partitions matters. That difference is why the split pays for itself.

How to read the SQL. PARTITION BY RANGE (started_at) says rows are split into chunks by start time, and each CREATE TABLE ... PARTITION OF defines one day's chunk. The primary key includes started_at because PostgreSQL requires a partitioned table's unique key to contain the partition column. started_at::date casts a timestamp to its calendar date.

Trade-offs and pitfalls

• Rollups add a job to maintain and can drift from raw; a nightly check comparing rollup totals to raw counts catches this.
• Raw retention is a cost decision: keep full attempts for a fixed window, then keep rollups plus failed attempts if long history matters.
• Too many indexes on a write-heavy table slow ingestion. Add the per-test index because history is a core query, and resist adding more until a query needs it.

Direct answer

Test isolation means one test's setup, execution, and teardown cannot affect another test's outcome; it matters because a suite where tests interfere with each other produces failures that depend on run order or which tests happened to run before it, which destroys the ability to trust a single test's result in isolation.

Structured elaboration

Common sources of interference in a microservice's test suite:

• Shared mutable state: a database, an in-memory cache, or a static/global variable that one test writes to and another test reads, so the second test's outcome depends on what the first test did.
• External resources with real state: a shared file, a shared queue, or a shared third-party sandbox account whose state persists across test runs.
• Nondeterministic inputs: the current wall-clock time or a source of randomness that produces a different value each run, so the same test can pass or fail depending on when or how many times it's run.
• Order dependence: a test that only passes because an earlier test happened to leave the system in a particular state, and fails if run alone or in a different order.

Practices to enforce isolation in CI: give each test (or each test worker, for parallel execution) its own database transaction that gets rolled back, or its own ephemeral schema/namespace; inject the clock and any randomness sources instead of reading them from the environment directly, so tests can pin them; run tests that must touch shared infrastructure serially or with explicit locking rather than assuming parallel safety by default; and treat "passes alone but fails in the full suite" as a bug in the test, not a fluke to re-run away.

Worked example

A microservice's order-processing tests all write to the same test database table without transactions. Test A creates an order with id 1001; test B, checking "creating a duplicate order id fails," happens to reuse 1001 and only passes because test A ran first and left that row behind. Wrapping each test in a transaction that rolls back at teardown, or generating a fresh unique id per test, removes the order dependence entirely: each test now sets up exactly the state it needs and leaves nothing behind for the next one.

Trade-offs and pitfalls

Enforcing isolation has a real cost (transactions, ephemeral environments, injected clocks add setup code), and it is tempting to skip it while the suite is small. The cost of NOT doing it compounds: as the suite grows, order-dependent tests accumulate silently until a routine reordering (parallelizing the suite, or adding a new test earlier in the file) causes a wave of failures with no obvious cause, which is far more expensive to untangle after the fact than to prevent up front.