Mocking, Stubbing, and Test Isolation Questions

Direct answer

Build a small Express server that keeps an in-memory map of configured per-id responses, exposes a control endpoint to set (and another to clear) that configuration at runtime, and serves whatever was last configured for a given id on GET /users/:id, returning a clear "not configured" response otherwise.

Structured elaboration

const express = require('express');

function createMockUserServer() {
  const app = express();
  app.use(express.json());
  const responses = new Map();

  app.get('/users/:id', (req, res) => {
    const configured = responses.get(req.params.id);
    if (!configured) {
      return res.status(404).json({ error: 'no mock configured for this id' });
    }
    res.status(configured.status).json(configured.body);
  });

  app.post('/__mocks', (req, res) => {
    const { id, status, body } = req.body;
    if (!id || !status) {
      return res.status(400).json({ error: 'id and status are required' });
    }
    responses.set(id, { status, body: body ?? {} });
    res.status(201).json({ ok: true });
  });

  app.delete('/__mocks/:id', (req, res) => {
    responses.delete(req.params.id);
    res.status(204).end();
  });

  return app;
}

module.exports = { createMockUserServer };

Test code using supertest (executed):

const request = require('supertest');
const { createMockUserServer } = require('./mock-server');

function assertEqual(actual, expected, msg) {
  if (actual !== expected) {
    throw new Error(`FAIL: ${msg} (expected ${expected}, got ${actual})`);
  }
}

async function main() {
  const app = createMockUserServer();

  let res = await request(app).get('/users/1');
  assertEqual(res.status, 404, 'unconfigured id should 404');

  res = await request(app).post('/__mocks').send({ id: '1', status: 200, body: { id: '1', name: 'Ada' } });
  assertEqual(res.status, 201, 'configuring a mock should return 201');

  res = await request(app).get('/users/1');
  assertEqual(res.status, 200, 'configured id should return the configured status');
  assertEqual(res.body.name, 'Ada', 'configured id should return the configured body');

  await request(app).post('/__mocks').send({ id: '2', status: 503, body: { error: 'unavailable' } });
  res = await request(app).get('/users/2');
  assertEqual(res.status, 503, 'configured error status should be honored');

  res = await request(app).delete('/__mocks/1');
  assertEqual(res.status, 204, 'reset should return 204');
  res = await request(app).get('/users/1');
  assertEqual(res.status, 404, 'reset id should 404 again');

  console.log('ALL 7 ASSERTIONS PASSED');
}

main().catch((e) => { console.error(e); process.exit(1); });

Executed with node: ALL 7 ASSERTIONS PASSED (the earlier draft referenced an assertEqual helper without defining it and never invoked main(), so it silently produced no output at all; this version defines the helper, calls main(), and the real count of assertions actually made is 7, not 5).

Worked example

A UI test that needs GET /users/42 to return a specific plan tier first calls POST /__mocks with { id: "42", status: 200, body: { id: "42", plan: "pro" } }, then exercises the page under test, then (in teardown) calls DELETE /__mocks/42 so the next test starts from a clean, unconfigured state rather than inheriting whatever the previous test left behind.

Trade-offs and pitfalls

The in-memory Map means all configured mocks are shared across whatever tests are running against this one server instance; if tests run in parallel against a single shared server process, they need distinct ids or their own server instance per test, otherwise one test's configured response can leak into another's assertions, the exact interference test isolation is meant to prevent. Explicitly resetting configured mocks between tests (or spinning up a fresh server per test) avoids this at the cost of some setup overhead per test.

Direct answer

Drive a controllable stub HTTP client through a scripted sequence of timeouts, slow responses, and server errors, then assert on both the retry count actually used and the circuit breaker's state transitions (closed to open, and open to half-open to closed again after a cooldown), rather than only checking the final result.

Structured elaboration

import time

class TransientError(Exception): pass
class CircuitOpenError(Exception): pass

class StubHttpClient:
    def __init__(self, script):
        self.script = list(script)
        self.calls = 0
    def get(self, path):
        self.calls += 1
        behavior = self.script.pop(0) if self.script else "ok"
        if behavior == "timeout":
            raise TimeoutError(f"timed out calling {path}")
        if behavior == "500":
            raise TransientError(f"server error calling {path}")
        return {"status": 200, "path": path}

class CircuitBreaker:
    def __init__(self, failure_threshold=3, cooldown_seconds=30, clock=time.monotonic):
        self.failure_threshold = failure_threshold
        self.cooldown_seconds = cooldown_seconds
        self.clock = clock
        self.state = "closed"
        self.consecutive_failures = 0
        self.opened_at = None

    def before_call(self):
        if self.state == "open":
            if self.clock() - self.opened_at >= self.cooldown_seconds:
                self.state = "half_open"
            else:
                raise CircuitOpenError("circuit is open")

    def record_success(self):
        self.consecutive_failures = 0
        self.state = "closed"

    def record_failure(self):
        self.consecutive_failures += 1
        if self.state == "half_open" or self.consecutive_failures >= self.failure_threshold:
            self.state = "open"
            self.opened_at = self.clock()

def call_with_retry(client, breaker, path, max_retries=2):
    breaker.before_call()
    attempts_used = 0
    while True:
        try:
            result = client.get(path)
            breaker.record_success()
            return result, attempts_used
        except (TimeoutError, TransientError):
            if attempts_used >= max_retries:
                breaker.record_failure()
                raise
            attempts_used += 1

class FakeClock:
    def __init__(self, start=0.0):
        self.now = start
    def __call__(self):
        return self.now
    def advance(self, seconds):
        self.now += seconds

def test_retries_then_succeeds_records_correct_retry_count():
    client = StubHttpClient(["timeout", "500"])
    breaker = CircuitBreaker(failure_threshold=3, cooldown_seconds=30, clock=FakeClock())
    result, retries_used = call_with_retry(client, breaker, "/thing", max_retries=2)
    assert client.calls == 3
    assert retries_used == 2
    assert result == {"status": 200, "path": "/thing"}
    assert breaker.state == "closed"

def test_circuit_opens_after_threshold_consecutive_failures():
    clock = FakeClock(start=0.0)
    client = StubHttpClient(["500", "500", "500", "ok"])
    breaker = CircuitBreaker(failure_threshold=3, cooldown_seconds=30, clock=clock)

    for _ in range(3):
        try:
            call_with_retry(client, breaker, "/thing", max_retries=0)
            assert False, "expected TransientError to propagate"
        except TransientError:
            pass

    assert breaker.state == "open"
    calls_before_reject = client.calls

    try:
        call_with_retry(client, breaker, "/thing", max_retries=0)
        assert False, "expected CircuitOpenError while circuit is open"
    except CircuitOpenError:
        pass
    assert client.calls == calls_before_reject  # rejected without ever reaching the stub client

    clock.advance(31)
    result, _ = call_with_retry(client, breaker, "/thing", max_retries=0)
    assert result == {"status": 200, "path": "/thing"}
    assert breaker.state == "closed"

if __name__ == "__main__":
    test_retries_then_succeeds_records_correct_retry_count()
    test_circuit_opens_after_threshold_consecutive_failures()
    print("ALL ASSERTIONS PASSED")

Executed with python3: both test_retries_then_succeeds_records_correct_retry_count and test_circuit_opens_after_threshold_consecutive_failures pass, printing ALL ASSERTIONS PASSED, with an injected FakeClock used to advance past the cooldown window deterministically (no real 30-second wait in the test).

Worked example

The retry test scripts the stub to fail twice (a timeout, then a server error) before succeeding, and asserts exactly 3 calls were made and the retry loop used 2 retries, proving the retry count is exact rather than "at least some retries happened." The circuit-breaker test scripts 3 consecutive failures, asserts the breaker transitions to open, then asserts a call attempted before the cooldown elapses is rejected WITHOUT even reaching the stub client (client.calls doesn't increase), which is the behavior that actually protects a struggling downstream dependency from being hammered while it's unhealthy. Advancing a fake clock past the cooldown and scripting a success proves the breaker correctly moves to half-open and then closes again.

Edge cases

Exponential-backoff overflow: as retries increase, a naive delay = base * 2^attempt can grow unbounded; the retry loop should cap the delay at a sane maximum rather than letting it grow indefinitely on a long failure streak. Idempotency: a retried call must not have a different effect than a single call, if the underlying operation isn't naturally idempotent (like a payment charge), the retry wrapper needs an idempotency key or another safeguard so retrying a timed-out call can't result in the operation happening twice.

Trade-offs and pitfalls

Asserting only on the final return value ("it eventually succeeded") would miss a broken retry count (say, retrying 10 times instead of the intended 2) or a broken circuit breaker that never actually opens; asserting on client.calls and on breaker.state directly is what actually verifies the RESILIENCE mechanisms, not just the happy path they're protecting. Using a real clock and a real time.sleep in this kind of test would make it either slow (waiting out real cooldowns) or flaky (racing against real timing); injecting the clock is what makes the cooldown-based state transition deterministic and fast.

Direct answer

Three approaches intercept network traffic during a Selenium test at different layers: an HTTP proxy sits between the browser and the network and can rewrite any request/response; browser-level network interception (via the Chrome DevTools Protocol) hooks directly into the browser's own network stack; and a service-virtualization tool replaces the backend entirely with a configurable fake server the browser talks to normally.

Structured elaboration

A short example stubbing a single JSON API response (shown here with an HTTP-level interception library so it is genuinely runnable without a browser; the identical idea applies whether the interception happens via a proxy or the Chrome DevTools Protocol):

import responses
import requests


def fetch_profile(user_id):
    resp = requests.get(f"https://app.example.com/api/profile/{user_id}")
    resp.raise_for_status()
    return resp.json()


@responses.activate
def test_stub_a_single_json_api_response():
    responses.add(
        responses.GET,
        "https://app.example.com/api/profile/77",
        json={"id": 77, "plan": "pro"},
        status=200,
    )

    result = fetch_profile(77)

    assert result == {"id": 77, "plan": "pro"}
    assert len(responses.calls) == 1

Executed with pytest:

test_s13_network_interception.py::test_stub_a_single_json_api_response PASSED
1 passed in 0.03s

Worked example

Testing a profile page that should show a "Pro" badge only for pro-tier users: stub the GET /api/profile/77 call to return {"plan": "pro"} and assert the badge renders; stub the same endpoint to return {"plan": "free"} in a second test and assert the badge does not render. Neither test depends on a real backend being up, and both are deterministic regardless of what the real profile service currently returns for user 77.

Trade-offs and pitfalls

An HTTP proxy adds a genuinely separate moving part (the proxy process itself, HTTPS certificate trust) to the test environment; browser-level interception avoids that but is coupled to whichever browser engine's devtools protocol you're using; service virtualization is the heaviest but also the most representative of real end-to-end behavior. Whichever layer is chosen, the stubbed response shape needs to be kept realistic, if the real API adds a required field the stub never includes, the UI test can pass while the real integration is actually broken.

Direct answer

A production bug escaped because an over-specified mock hid the real integration behavior; the right response is to name that concrete failure as the trigger, then propose a phased, measured plan (add real integration and contract tests, tighten what mocks are allowed to assert, and track defect-escape metrics) with an explicit rollback path if the change itself causes disruption.

Structured elaboration

• Lead with the trigger, not the abstraction: a stakeholder-facing proposal lands better grounded in the specific incident (what broke, why the existing mock-heavy tests didn't catch it) than in a general claim that "our tests are too mock-heavy."
• Cost/benefit case: quantify what the incident cost (customer impact, engineering time to diagnose and fix) against the ongoing cost of the proposed change (slower test suite, engineering time to build real/contract tests), so the trade-off is concrete rather than asserted.
• Phased technical plan over roughly a quarter: start by adding a small number of real integration tests or consumer-driven contract tests specifically covering the class of interaction that broke; in parallel, audit existing mocks for over-specification (asserting on internal call details rather than observable behavior) and simplify the worst offenders; avoid attempting to rewrite the entire suite's mocking strategy at once.
• Rollback plan: if the added integration/contract tests turn out to be too slow or flaky to run on every commit, have a fallback (running them on a slower cadence, or gating only specific high-risk services) rather than reverting the whole initiative; state this explicitly so the proposal doesn't read as an irreversible, high-risk bet.
• Metrics for success: track the defect-escape rate (bugs that reach production versus caught pre-merge) and integration-test coverage growth over the quarter, reporting back against these numbers rather than declaring success by feeling.
• Migration steps that acknowledge different audiences: individual contributors need concrete guidance on which mocks to simplify and how to write a new contract test; engineering leadership needs the cost/benefit framing and the metrics; both need to see this connected back to the specific incident that motivated it.

Worked example

After a mock hid a change in a downstream service's error-response shape (the mock always returned the old shape, so the calling code's new error-handling branch was never actually exercised, and shipped broken), the proposal names that incident explicitly, then commits to: adding 3 consumer-driven contract tests for that specific downstream service in month one, auditing and simplifying the 12 most over-specified mocks in month two, and reporting defect-escape rate and contract-test coverage at the end of the quarter. If the contract tests prove too slow for the standard per-commit pipeline, the rollback path moves them to a nightly gate rather than abandoning them.

Trade-offs and pitfalls

A proposal that argues from principle ("mocking is a code smell") without anchoring in the specific incident and its concrete cost is a harder sell to stakeholders who weigh engineering time against other priorities; leading with the real trigger event, the real cost, and a bounded, reversible phased plan is what actually gets this kind of initiative funded and, more importantly, followed through on rather than abandoned halfway.

Direct answer

Threads, async/await, and separate processes each threaten test isolation in a different way: threads share memory directly and can race on any shared mutable state, async/await tasks share memory too but only yield at explicit await points (making races narrower but still real), and separate processes don't share memory at all but can still interfere through shared external resources like a database or a port.

Structured elaboration

• Threads: the highest interference risk, since any shared mutable object can be read and written from multiple threads at truly arbitrary points, including partway through a non-atomic operation. A parallel test runner using threads needs each test's data either fully isolated (no shared objects) or protected by synchronization, and must watch for memory leaks from threads that outlive the test that spawned them.
• Async/await (single-threaded concurrency): interleaving only happens at await points, since code between awaits runs to completion without preemption, which narrows the surface for races but doesn't eliminate it, two async tasks can still race on a shared variable if one awaits in the middle of a read-modify-write sequence the other also touches.
• Separate processes: memory isn't shared at all, which removes in-memory races entirely, but tests running in separate processes can still interfere through anything they share externally, the same database, the same listening port, the same temp directory, or the same external service account.

Designing a parallel test runner that avoids shared-state issues: give each test (or each parallel worker) its own isolated resources wherever possible, an in-memory or per-worker database, dynamically-assigned ports instead of hardcoded ones, per-worker temp directories, and be deliberate about fixture scope so an expensive-to-create resource that IS shared across tests in a worker is either read-only or explicitly protected.

Deciding which tests are safe to parallelize: a test is safe to run in parallel with others if it doesn't depend on or mutate anything the other parallel tests can observe, tests touching genuinely exclusive external resources (a fixed port, a singleton external sandbox account) should stay serial or get resource-level locking rather than being forced into parallel execution.

Worked example

A parallel test runner using worker processes (not threads) for isolation still saw intermittent failures because two workers both tried to bind port 5000 for a local test server; the fix was to have each worker request an OS-assigned ephemeral port instead of hardcoding one, removing the shared external resource that had been silently coupling otherwise fully isolated processes.

Trade-offs and pitfalls

Choosing process-based parallelism over thread-based parallelism trades away in-memory races for higher per-worker overhead (each process needs its own runtime and memory), it's a legitimate trade for many test suites but not free. A parallel runner design that assumes "no shared memory" automatically means "no interference" misses the port/database/file-based interference case entirely, which is exactly the kind of gap that produces intermittent, hard-to-reproduce parallel-test failures.