Microsoft SDET (Software Development Engineer in Test) - Mid Level Interview Preparation Guide

Direct answer: The three policies trade release safety for developer velocity along a spectrum, immediate quarantine favors velocity and visibility, fail-on-flake favors correctness at the cost of blocking unrelated work, and skip-with-annotation is the cheapest but weakest on both signal and accountability, so the right choice depends on how much a company can tolerate either blocked releases or silently degraded coverage.

Structured elaboration

Recommendations by company context:

• Startup: skip-with-annotation or lightweight quarantine, moving fast matters more than perfect suite hygiene at this stage, and the team is small enough that "someone remembers" can substitute for formal SLA enforcement, at least temporarily. The risk is this doesn't scale past a certain team size without becoming quarantine's downside (an ever-growing, unmanaged skip list).
• Mid-size company: quarantine with a strict, enforced SLA is usually the best balance, enough scale that fail-on-flake's blocking cost becomes prohibitive, but enough process maturity to sustain an ownership/SLA system without it collapsing into skip-with-annotation's laissez-faire failure mode.
• Regulated enterprise: fail-on-flake, or quarantine with MUCH stricter oversight (a compliance-reviewed, audited quarantine list rather than a lightweight engineering process), since the cost of a real regression shipping (potential compliance or safety consequences) generally outweighs the velocity cost of blocking merges, and audit requirements often demand exactly the kind of accountability trail fail-on-flake naturally produces.

Worked example: a regulated financial-services company evaluated skip-with-annotation early on and found, in an internal audit eight months later, over 200 silently-skipped tests with no owner and no record of why they were skipped, several covering critical calculation paths, exactly the failure mode the policy comparison above predicts. They moved to fail-on-flake for anything touching a regulated calculation path, and quarantine-with-audit-trail for everything else, accepting the higher operational cost of faster triage capacity as the price of the compliance posture they needed.

Trade-offs & pitfalls: it's tempting to pick one policy for the whole organization, but the worked example shows the more common real-world pattern is a HYBRID, different policies for different risk tiers of test (a payments-critical test gets fail-on-flake; a low-risk internal-tool test gets quarantine or skip). A pure one-size-fits-all policy either over-constrains low-risk work (unnecessary fail-on-flake friction everywhere) or under-constrains high-risk work (skip-with-annotation on something that actually matters).

These names describe two different axes, not one: smoke, regression, integration, system, and user-acceptance tests describe SCOPE and PURPOSE within a release process, while functional versus non-functional describes WHAT KIND of requirement is being verified. A single test can sit at one point on each axis at once (for example, a load test is a non-functional system test).

The five scope/purpose types

Regression testing's specific effect on release velocity

A solid regression suite is what lets a team ship frequently without re-manually-verifying everything that already worked: automated regression tests reliably catch a bug like the double-discount example above, where a change to one part of the pricing logic silently breaks a previously-correct interaction, the moment it's introduced, rather than after a customer reports it. What automated regression tests do NOT reliably catch is a bug that requires actual human judgment to notice, such as a new promotional banner rendering with confusing or misleading wording, which passes every automated check while still being wrong; that class of issue needs manual exploratory testing precisely because "correctness" here is a judgment call, not a fixed assertion.

Keeping a growing regression suite fast and reliable

As a regression suite grows, two problems compound: it gets slower, and it accumulates flaky tests (ones that fail intermittently for reasons unrelated to real regressions). Keep it fast by running only the subset of regression tests relevant to changed code on every PR, reserving the full suite for a nightly run. Keep it reliable by treating a flaky regression test as a bug in the test itself, not background noise to tolerate: track a rerun rate per test, and either fix or quarantine (temporarily exclude with an owner assigned to repair it) any test whose failures don't correlate with real code changes, since an ignored flaky test trains the team to distrust the whole suite.

Functional versus non-functional testing, as a separate axis

Functional testing asks "does the checkout flow correctly compute the total and complete the order," a direct check against a stated feature requirement. Non-functional testing asks a different kind of question entirely: for the same checkout flow, does it perform well under load (performance), does it protect payment data appropriately (security), is it usable by someone unfamiliar with the site (usability), and can someone using a screen reader complete a purchase (accessibility). These four non-functional concerns should be prioritized before release based on business risk, not treated as equally weighted: for a payment flow specifically, security and performance under peak load typically deserve the most pre-release attention, since a failure there has the most severe and hardest-to-reverse consequences, while usability and accessibility issues, though real and important, are more often caught and improved iteratively after release without the same acute risk.

In a typical CI/CD pipeline, functional tests run continuously as part of the regular suite on every commit or pull request, since a functional regression, like the checkout total being computed incorrectly, is valuable to catch immediately. Non-functional tests usually run on a slower, scheduled cadence: a load test simulating peak Black Friday traffic against the checkout API is a concrete non-functional example, and it typically runs nightly or pre-release rather than blocking every commit, since it needs a longer, resource-heavy run that would slow down PR feedback if it gated every merge.

Trade-offs and pitfalls

The most common confusion is treating "system test" and "end-to-end test" as interchangeable; they overlap heavily in practice but system testing traditionally emphasizes validating the WHOLE application against its requirements as one unit (often owned by QA, closer to release), while end-to-end testing more narrowly emphasizes a specific user JOURNEY through the real stack (often automated and run continuously). Naming this distinction explicitly, rather than treating the terms as synonyms, is itself a signal of depth in this space.

Direct answer

With 10,000 unit tests averaging 0.8 seconds each, the total sequential work is 8,000 test-seconds; against a 2-minute (120-second) wall-clock SLA (service-level agreement, the target you've committed to) under ideal parallelization, that requires ceil(8000 / 120) = 67 parallel runners. In practice you'd need meaningfully more than 67 once real-world factors are accounted for.

Structured elaboration

The math: $\text{total work} = 10{,}000 \times 0.8\text{s} = 8{,}000\text{s}$. Runners needed under perfect parallelization: $\lceil 8000 / 120 \rceil = 67$ (since $8000/67 \approx 119.4\text{s}$, just under the 120-second SLA, while 66 runners would give $8000/66 \approx 121.2\text{s}$, just over it).

Practical factors that push the real number higher than 67:

• Runner start-up time: if each runner takes, say, 15-20 seconds to become ready, that's a meaningful fraction of a 120-second budget eaten before any test even runs, effectively shrinking the usable window and requiring more runners to compensate.
• Test-runner warm-up: the first test on a freshly started runner (JIT warm-up, cold caches, first-connection latency) is often slower than steady-state, which the naive average-duration calculation doesn't capture.
• Imperfect balancing: 10,000 tests divided across 67 runners won't split perfectly evenly in practice (some runners will get a slightly heavier share, especially if durations vary rather than being uniformly 0.8s), so the slowest runner, not the average, determines whether you hit the SLA.
• Cost trade-offs with hourly-billed VMs: cloud VMs billed by the hour (or with a minimum billing increment) mean provisioning 67 runners for a 2-minute job may still incur close to a full hour's cost per runner depending on the provider's billing granularity, so the "ideal" runner count and the cost-optimal runner count can diverge; a team might deliberately provision somewhat below the ideal-latency number and accept a slightly longer SLA to avoid paying for mostly-idle billed-hour capacity.

Worked example

The exact calculation was verified: $10{,}000 \times 0.8 = 8000$ total work-seconds; $\lceil 8000/120 \rceil = 67$ runners, confirmed by checking $8000/67 \approx 119.4\text{s} \le 120\text{s}$ while $8000/66 \approx 121.2\text{s} > 120\text{s}$. Accounting for a realistic 15-second runner start-up overhead eating into the budget (leaving roughly 105 usable seconds per runner instead of 120), the practical requirement rises to $\lceil 8000/105 \rceil = 77$ runners, about 15% more than the naive ideal-parallelization figure.

Trade-offs & pitfalls

The most common mistake is stopping at the naive ideal-parallelization number (67 here) and provisioning exactly that, without accounting for start-up overhead and imbalance, which reliably causes the SLA to be missed in practice even though the math "checked out"; building in a deliberate margin (provisioning for the effective, not the naive, usable time window) is what actually meets the SLA consistently.

Direct answer

A credible estimate for a bounded piece of owned technical work is never a single number. It is a range built from a real decomposition of the work, with the assumptions and risk factors named out loud rather than folded silently into padding. Strong candidates separate "the work I can size confidently" from "the unknowns that could blow this up" and present both.

Structured elaboration

1. Define done first. Before sizing anything, pin down what "done" actually means (what gets tested, what gets deployed, who signs off). An estimate against a fuzzy definition of done is not an estimate, it is a guess.
2. Decompose into small stages. Break the work into pieces small enough that each one is individually estimable (roughly a day to a few days each), not one lump. Small stages make errors easier to catch and let you re-forecast partway through instead of waiting until the end to discover you were wrong.
3. Name your assumptions explicitly. Write down what you are assuming about scope, existing tooling, data availability, and team support. These assumptions are exactly what turns out to be wrong later, so writing them down is what lets you catch it early.
4. Identify the risk factors that could blow up the estimate, separately from the baseline work itself: unfamiliar technology, an external dependency you do not control, environment or data-access delays, and any stage that depends on something you have not verified yet. The biggest risk to a technical estimate is almost always one of these, not the raw coding effort.
5. Estimate each stage as a range (an optimistic case if nothing goes wrong, and a likely case), then add contingency sized to the risk factors you named, rather than a flat percentage applied blindly everywhere.
6. Present timeline, resourcing, and confidence as one package: the stage breakdown, the range with a stated confidence level, the top two or three named risks that could move the number, and the checkpoint at which you will re-confirm the estimate once the biggest unknown is retired.

Worked example

Take automating a regression suite. Decomposition: audit the existing manual suite and select which cases to automate (2 days), extend the automation framework (3 days), author automated tests for 30 selected cases at roughly half a day each (15 days), stabilize flaky tests and wire into CI (continuous integration, the automated pipeline that runs tests on every code change) (3 days). Baseline sum: 2 + 3 + 15 + 3 = 23 days.

The two named risk factors: the automation framework may need real rework if the underlying application's UI changes mid-project, and test-environment access could be delayed. Contingency for those two risks adds roughly 3 days, giving a likely estimate of 26 days. Optimistic case (framework needs no rework, stabilization goes smoothly): 21 days. Pessimistic case (environment access is delayed by a full week): 26 + 5 = 31 days.

What I would present to stakeholders: "roughly 5 to 6 weeks (about 26 working days) at medium confidence, with a realistic range of 21 to 31 days depending on two named risks: framework rework and environment access. I will check back in at day 10, once the framework and audit stages are done, and re-confirm or revise the range."

The same shape applies just as directly to a reporting-dashboard build for a Business Intelligence or Data Analyst: decomposition might run source-data validation (2 days), query and metric-definition build (4 days), visual build and stakeholder review cycles (5 days), and a final accuracy reconciliation against a known source of truth (2 days), for a baseline of 13 days. The named risk factors shift to data-quality gaps discovered during reconciliation and a slow stakeholder review turnaround, rather than framework rework, but the same discipline holds: present a range with a stated confidence level, name the top risks, and set a checkpoint (for example, after the query and metric-definition stage) to re-confirm the estimate rather than wait until the deadline.

Trade-offs and pitfalls

Padding the number silently instead of naming the actual risk teaches stakeholders nothing about what to watch. Giving a single point estimate as if it were a fact invites a commitment you cannot actually make good on. Not re-forecasting at a natural checkpoint means the first time anyone learns the estimate was wrong is at the deadline. And anchoring the estimate to the requester's preferred date, then reverse-engineering a plan to fit it, is the fastest way to lose credibility the next time you estimate anything.

Direct answer

Introduce testability into legacy code by finding or creating a "seam", a point where you can substitute a different behavior without editing the class's own logic, typically by extracting an interface around a hard-coded dependency, wrapping a global or static call behind an injectable object, or using a language's interception features as a last resort.

Structured elaboration

A practical, risk-minimizing sequence:

1. Add characterization tests first: before changing any production code, write tests that pin down the class's CURRENT behavior (even if that behavior isn't ideal), using whatever coarse-grained testing is possible today (an end-to-end call, or testing through the public API). These act as a safety net so later refactors can be verified not to change behavior.
2. Find the smallest seam: look for a hard-coded new SomeDependency() call, a static method call, or direct file/network access buried inside the class, and identify the minimal change that would let a test substitute something else there.
3. Extract an adapter/interface around that dependency: wrap the concrete class behind a small interface that mirrors only the methods actually used, then have the legacy class depend on the interface instead of the concrete implementation.
4. Introduce a wrapper for globals or statics: if the dependency is a static method call or a global singleton, wrap it in a thin instance-based adapter class that a test can inject a fake implementation of, since most languages can't mock static/global calls directly without special tooling.
5. Only reach for runtime interception (bytecode manipulation, monkeypatching) when extraction is genuinely impractical, since it works without changing production code but produces tests that are more fragile and harder for future readers to understand than an explicit seam.
6. Grow real unit test coverage behind the new seam, then repeat the process on the next hard-coded dependency, expanding coverage incrementally rather than attempting one large rewrite.

Worked example

A legacy InvoiceGenerator class calls new PdfLibrary().render(invoice) directly inside a large generate() method. To make it testable: extract a PdfRenderer interface with a render(invoice) method, have the concrete PdfLibrary-backed implementation implement it, and change InvoiceGenerator to accept a PdfRenderer through its constructor instead of constructing PdfLibrary itself. Before making this change, a characterization test calls the existing generate() method with a few representative invoices and asserts on today's actual PDF output (or a hash of it), so if the refactor accidentally changes behavior, that test catches it immediately, even though it doesn't yet use any test doubles.

Trade-offs and pitfalls

The temptation on a large legacy class is to introduce many seams at once; doing that without characterization tests in place first risks silently changing behavior during the very refactor meant to make the class safer to change. Runtime interception techniques (monkeypatching a language runtime, bytecode-rewriting a "final" class) can unblock testing fast, but they tend to make the resulting tests fragile and harder to reason about than an explicit extracted seam, so treat them as a stopgap on the path to a real interface, not the end state.

Direct answer
A peer-review checklist for test cases exists to catch the specific ways a test case can be technically present but still worthless: ambiguous enough that two engineers would execute it differently, coupled to another test's leftover state, dependent on manual judgment to pass, disconnected from the requirement it's supposed to verify, or missing the edge cases that were the actual reason the feature was risky in the first place; the checklist below covers clarity, independence, repeatability, traceability, edge-case coverage, expected-result quality, data setup/cleanup, and automation-readiness, each with a concrete reviewer question and a one-line rationale.

Structured elaboration: the checklist

Worked example
Applying items 6, 7, and 9 to a single real test case under review: a submitted case for "delete user account" reads "Steps: click delete, confirm. Expected: account is deleted." A reviewer applying this checklist would flag it on item 6/7 (deleted from WHERE, exactly: does the user record get hard-deleted, soft-deleted with a flag, or anonymized? Does the response return a specific status code? Are related records, like that user's orders, deleted, retained, or reassigned?) and on item 9 (no teardown is stated, but a permanently-deleted test account needs a documented recreation step for the next test run, or the environment accumulates one fewer usable test account every time this case runs).

Trade-offs and pitfalls
The most common wrong turn in applying a checklist like this is treating every item as equally weighted and equally mandatory for every test case; a low-risk, purely cosmetic UI test genuinely doesn't need the same edge-case-coverage rigor as a payment or authentication test case, and a reviewer who mechanically demands full rigor everywhere trains authors to game the checklist with box-ticking rather than genuinely improving the highest-risk cases. The other pitfall is using the checklist only at review time and never during authoring; a checklist that's purely reactive catches problems after the effort of writing the case is already sunk, while checking against it while WRITING the case (especially items 4, 6, and 9) is cheaper to fix and produces fewer review round-trips.

Direct answer

Measuring automation ROI over a genuine 6-month experiment, rather than a one-time estimate, means instrumenting three kinds of data from the start (manual hours avoided, maintenance hours spent, and defect-detection rate), collecting them consistently across the period, and presenting the trend, not just an endpoint number, to a non-technical stakeholder.

Structured elaboration

Metrics to collect:

• Manual-hours avoided: for each automated test, track how many manual execution hours it would have taken if run by hand at the same frequency, computed from historical manual execution time before automation.
• Maintenance-hours spent: track actual engineer time spent fixing, updating, or investigating failures in the automated suite, since this is the real ongoing cost side of the ROI equation, not a one-time estimate. A separate one-time figure, the initial hours spent building the suite in the first place, should also be tracked and netted out of the ROI calculation once, at the start, since conflating it with ongoing maintenance-hours or omitting it entirely is what makes an automation investment look like it paid back faster than it actually did.
• Defect-detection rate: track how many real defects the automated suite catches over the period, ideally distinguishing defects caught pre-release by automation from those that still escaped to production, to show whether automation is actually improving detection, not just replacing manual labor with equivalent detection power.

Instrumenting data collection: tag each automated test with metadata (what manual process it replaced, an estimated manual-time-per-run baseline) at the time it is built, log every test run's pass/fail result and, when it fails due to a genuine defect versus a flaky or maintenance issue, classify which; track engineer time spent on automation maintenance through existing time-tracking or ticket-tagging practices rather than trying to build new tooling from scratch.

Presenting to a non-technical stakeholder: a simple monthly trend line showing cumulative manual-hours avoided against cumulative maintenance-hours spent, crossing over at the point of positive net ROI, plus a small number of concrete examples of real defects the automation caught that would plausibly have reached production otherwise, since specific, vivid examples land better with a non-technical audience than an aggregate percentage alone.

Worked example

Over 6 months, monthly data might show (in addition to a one-time initial build investment of roughly 50 engineer-hours spent before month 1 to construct the automated suite, a cost distinct from the ongoing maintenance-hours metric and one that must be netted out too, or the ROI story overstates how fast the investment actually pays back): month 1, 40 manual-hours avoided against 15 maintenance-hours (early stage, more setup and fixing); month 2, roughly 60 manual-hours avoided against 14 maintenance-hours; by month 3, 90 manual-hours avoided against 12 maintenance-hours as the suite stabilizes; months 4 and 5 continuing the same trend at roughly 115 and 128 manual-hours avoided against 11 and 10 maintenance-hours; by month 6, 140 manual-hours avoided against 10 maintenance-hours as the suite matures and requires less upkeep. Netting each month's avoided-hours against that month's maintenance-hours, and starting from negative 50 to account for the initial build investment, gives a running cumulative total of negative 25 (month 1, still negative once the build investment is included), positive 21 (month 2, now crossing into positive territory), positive 99 (month 3), positive 203 (month 4), positive 321 (month 5), and positive 451 (month 6). Over the same period, the suite catches 8 real defects pre-release that a sample review confirms would likely have reached production under the previous manual-only process, including one specific, concretely described incident (a regression in discount-calculation logic caught the day it was introduced rather than discovered by a customer weeks later). Presented to a stakeholder as: "the investment paid for itself around month two once you count the time it took to build the suite in the first place, has saved roughly 450 cumulative engineer-hours over six months net of both maintenance and the initial build cost, and caught 8 real bugs before they reached customers, including this specific example."

Trade-offs and pitfalls

The most common mistake is measuring manual-hours avoided without also tracking maintenance-hours spent, which overstates the net benefit and sets an expectation the automation will not actually meet once real maintenance cost is accounted for. The second mistake is presenting only an aggregate defect count without concrete examples; a non-technical stakeholder connects much more strongly with "here is a specific bug this caught" than with a bare number.

Direct answer

Basic Auth and static API keys are simplest to test end-to-end since there's no token lifecycle involved. OAuth 2.0 bearer tokens and client TLS certificates are usually more practical to test with a mocked or pre-generated credential most of the time, reserving a real end-to-end login flow for a small number of specific tests, because the full flow is slower, harder to make deterministic, and often depends on infrastructure your test suite doesn't want to own.

Structured elaboration

Basic Auth. Test with a valid username/password pair, an invalid one, and a missing Authorization header, asserting 200, 401, and 401 respectively. Because there's no token to generate or expire, this is the simplest mechanism to test fully end-to-end every time, there's little benefit to mocking it.

API keys in headers. Similarly simple: a valid key returns success, an invalid or revoked key returns 401, a missing key returns 401. The main thing worth testing beyond that is whether the key is scoped (some keys can only call certain endpoints), which needs its own test asserting a valid-but-wrongly-scoped key returns 403, not 401, since 403 correctly signals "I know who you are, you're just not allowed to do this."

OAuth 2.0 bearer tokens. This is where mocking versus real end-to-end matters most. A full OAuth flow (redirect to an authorization server, user consents, authorization code exchanged for a token) is slow, involves a browser or a simulated one, and depends on a real or sandboxed identity provider being available. For the large majority of API tests, that's the wrong tool: instead, pre-generate a valid token (either from a test identity provider you control, or a token minted directly with a known signing key in a test environment) and use it directly as a fixture. Reserve actually driving the full OAuth flow for a handful of tests specifically about the LOGIN flow itself, not for every test that merely needs to be authenticated to check something else.

Client TLS certificates. Testing this properly generally needs an actual TLS handshake with a client certificate, since a mocked "you have a valid cert" check doesn't exercise the real validation logic: certificate chain validation (confirming the cert was actually issued by a certificate authority the server trusts, not just any self-signed cert), expiry checking (confirming the cert falls within its stated validity window), and revocation checking (confirming the cert hasn't been invalidated early by its issuer, typically via a certificate revocation list or OCSP, the Online Certificate Status Protocol, before its normal expiry date). This usually means a small number of tests running against a real TLS-terminating endpoint configured with a test certificate authority, rather than being mockable the way a bearer token is.

Trade-offs and pitfalls

Mock vs. full flow, decided by what's actually being tested. If the test's purpose is "does this endpoint correctly reject an expired token," you don't need a real OAuth flow to produce an expired token, you can mint one directly with an expired timestamp using the same signing key your test environment trusts, which is faster and more deterministic than trying to wait for a real token to expire. If the test's purpose is the login flow itself, redirect handling, consent screen, code exchange, then only a real (or realistically simulated) flow actually tests it; a mock would just assume the thing being tested works.

A common mistake is mocking authentication so completely that a real, deployed misconfiguration (a wrong signing key, an expired certificate, a misregistered redirect URI) would sail through every test undetected, because none of the tests ever exercised the real mechanism at all. The right balance keeps a small number of true end-to-end auth-flow tests as a canary, backed by a much larger number of tests that use pre-generated credentials to test everything else quickly.

Direct answer. The Page Object Model (POM) is a design pattern that wraps each page (or reusable UI region) in a class exposing locators and high-level actions, so tests call methods like login_page.login(user, pw) instead of touching selectors directly. It separates what a test wants to do from how the page currently implements it, so a UI change means editing one class, not every test that touches that page.

Structured elaboration.

• Responsibilities of a page object: hold the locators for that page/component, expose action methods (login, add_to_cart) and query methods (is_logged_in, get_price) that return plain data, never raw WebElement handles, so assertions stay in the test.
• Pros: one place to fix a broken locator; tests read like user actions instead of DOM manipulation; onboarding is faster because a new page's shape is discoverable from its class.
• Cons: a naive implementation adds an extra layer of indirection for very small suites; if page objects grow "god classes" covering unrelated flows, they become as brittle as the tests they replaced.
• Anti-patterns to avoid:
  1. Assertions inside the page object ("logic in POM") - a page object that calls assert couples it to one test's expectations and can't be reused by a different test that wants a different check.
  2. Locators or raw waits inside the test ("logic in tests") - defeats the whole point; the test should never see a CSS selector.
  3. Missing explicit waits - a page object that clicks immediately after navigation, with no wait for the target element, is the single most common source of flaky POM-based suites.
• Structuring page/component objects: for anything reused across pages (a nav bar, a product card, a modal), extract it as its own component-object class that a page composes by reference, rather than duplicating its locators/actions on every page that contains it. Locators, actions, and assertions get their own layer: locators are private constants, actions are public methods that use them, assertions live only in the test (or in an assertion helper the test calls).

Worked example. A minimal Python layout for a small login-flow project:

tests/
  test_login.py          # calls LoginPage methods, asserts on plain return values
pages/
  login_page.py          # class LoginPage: locators + navigate()/login()/error_message()
  base_page.py            # shared wait helpers all page objects inherit

class LoginPage(BasePage):
    USERNAME = ("id", "username")
    PASSWORD = ("id", "password")
    SUBMIT = ("id", "submit")
    ERROR = ("css selector", ".error-banner")

    def login(self, username, password):
        self.type_into(self.USERNAME, username)
        self.type_into(self.PASSWORD, password)
        self.click(self.SUBMIT)
        self.wait_until_visible(self.ERROR, optional=True)

    def error_message(self):
        return self.text_of(self.ERROR) if self.is_visible(self.ERROR) else None

test_login.py calls LoginPage(driver).login(...) and asserts on error_message() - it never sees a locator.

Trade-offs and pitfalls. POM is not free: over-abstracting a page object into a "framework within the framework" (generic perform(action_name, **kwargs) dispatchers) trades locator duplication for indirection nobody can trace during a failure. The senior judgment call is knowing when a shared component genuinely appears on multiple pages (extract it) versus when two pages only look similar today and will diverge (don't prematurely extract, or the shared class becomes a tangle of if page_type == ... branches).

Approach summary (SDET perspective)
Introduce performance tests as first-class jobs in the automation ecosystem but keep them logically separated from fast functional runs. Use the functional framework for test definitions where helpful, but run load/latency workloads on dedicated runners and infra to avoid noisy neighbors.

Architectural choices

• Separate runners (recommended): functional runner for unit/integration tests; dedicated performance runner(s) (k6/Gatling/JMeter) on isolated nodes or k8s pods. Pros: stable baselines, controllable resource allocation, easier scaling. Cons: additional infra.
• Same runner (only for lightweight latency/smoke): reuse framework harness to validate endpoints quickly; not suitable for high-concurrency load.

Resource isolation

• Run performance jobs on dedicated VMs/k8s namespaces with CPU/memory limits; use separate networks or subnets to emulate real topology.
• Use traffic shaping (tc/chaos/latency proxies) to measure client-side latency.
• Tag and pin runners to avoid co-located CI workloads; use spot instances for non-critical runs.

Data collection & aggregation

• Emit metrics from clients and target services to Prometheus via Pushgateway or exporters.
• Capture request/response times, error rates, throughput, system metrics (CPU, memory, GC, thread pools), and network stats.
• Store raw traces/logs in a time-series DB (Prometheus) and long-term storage (Elasticsearch/S3). Visualize in Grafana; keep run metadata in a test results DB for correlation.

Thresholding & failure semantics

• Define tiers: smoke latency gates (fast-fail), load thresholds (soft gates), and SLA breaches (hard fail).
• Use statistical thresholds (p95/p99) and error-rate limits. Example: fail build if p95 latency > 500 ms AND error rate > 1% for 2 consecutive runs.
• Implement anomaly detection and rollback rules for canaries; allow “warning” status that notifies but doesn’t block for flaky conditions.

CI gating strategies

• PR pipeline: unit + functional + lightweight performance smoke (single-user latency).
• Merge/master: nightly full load/stress runs; results posted to dashboards and issues created on regressions.
• Release/canary: run targeted performance suites against canary environment with strict pass/fail; gate production promotion on critical SLA metrics.
• Use progressive gating: require trending stability over N runs before blocking.

Cost & infra considerations

• Use auto-scaling k8s clusters and spot instances for large scale; schedule heavy runs off-hours.
• Right-size scenarios: synthetic population vs full-traffic replay; sample lower concurrency for CI and reserve full scale for nightly/regression windows.
• Reuse containers/images to reduce startup cost; archive artifacts to S3 to avoid repeated data transfer.

Example stack

• Test harness: pytest / Java test runner for definitions
• Load: k6 (scripts in repo) or Gatling
• Metrics: Prometheus + Grafana, ELK for logs
• Orchestration: GitHub Actions/Jenkins triggers, k8s for runners

Why this works: separates concerns, ensures repeatable baselines, provides fast feedback in PRs while enabling realistic, scalable stress testing with robust observability and controlled cost.