Test Automation Scripting Questions

Direct answer

A test fixture is the setup an automated test needs before it can run and the cleanup it needs afterward, whether that is a database connection, a browser instance, or a mock server; most test frameworks let you scope a fixture to run once per test, once per class of tests, or once per whole test suite/module, and choosing the right scope is a real trade-off between speed and isolation.

Structured elaboration

Common resources fixtures prepare and clean up: a database connection or transaction (opened before, rolled back or closed after), a browser/WebDriver session (launched before, quit()'d after), a mock server standing in for a third-party dependency (started before, stopped after), or simply test data seeded into a shared environment.

Scope is the key decision. Method-level (per-test) fixtures give the strongest isolation: every test gets a completely fresh resource, so nothing one test does can leak into another, at the cost of paying the setup cost (launching a browser, opening a connection) on every single test. Suite/module-level fixtures amortize that cost across many tests, which is much faster, but every test sharing that fixture must be written defensively enough not to depend on, or corrupt, state left behind by the tests that ran before it, across pytest specifically, scope='function' is the default (per-test), scope='class' shares across one test class, and scope='module'/scope='session' share across a whole file or the entire run respectively, in increasing order of both speed gained and isolation risk taken on.

Across the multiple frameworks a real team might use, the same idea appears under different names: pytest's @pytest.fixture, JUnit's @BeforeEach/@BeforeAll (paired with @AfterEach/@AfterAll), and TestNG's @BeforeMethod/@BeforeClass/@BeforeSuite, all express the same setup-scope-teardown shape.

A frequent real need is sharing a single WebDriver or database connection safely across many tests for speed: the safe pattern is a session- or module-scoped fixture for the EXPENSIVE, STATELESS part (launching the browser process itself, opening the connection pool) combined with a function-scoped fixture for anything STATEFUL that must not leak (navigating to a fresh page and clearing cookies before each test, or wrapping each test's database work in its own transaction that rolls back at the end), rather than sharing the stateful parts directly.

Worked example

import pytest

@pytest.fixture(scope="session")
def browser_process():
    # expensive, stateless: launch once for the whole run
    driver = launch_browser()
    yield driver
    driver.quit()

@pytest.fixture(scope="function")
def clean_page(browser_process):
    # cheap, stateful: reset to a known state before EVERY test
    browser_process.delete_all_cookies()
    browser_process.get("about:blank")
    yield browser_process

Here browser_process pays the expensive browser-launch cost once per test session, while clean_page guarantees every individual test starts from a clean slate (no leftover cookies or navigation state), combining the speed of session scope with the isolation function scope would otherwise provide alone.

Trade-offs and pitfalls

The most common mistake is sharing a session-scoped fixture that ALSO carries mutable state (a logged-in browser session, an open database transaction with uncommitted writes) directly across tests: the first test to run can leave the shared resource in a state the next test silently depends on, which produces order-dependent flakiness that only shows up when tests run in a different order or in parallel. Function scope avoids this entirely at the cost of speed; the combined pattern above (expensive+stateless at wide scope, cheap+stateful reset at narrow scope) is usually the better trade-off once a suite is large enough that full per-test setup is genuinely too slow.

Direct answer

Register two custom pytest CLI options (--base-url, --env) in conftest.py's pytest_addoption, then expose them to tests through fixtures that read request.config.getoption(...), with each fixture falling back to a sensible default when the flag is not passed; tests then simply depend on the base_url/env/credentials fixtures and never need to know whether the values came from a CLI override or the default.

Structured elaboration

pytest_addoption(parser) is a conftest.py hook pytest calls once at startup, before collecting any tests, giving you the chance to register new command-line flags via the same argparse-style parser.addoption(...) API pytest uses for its own built-in flags. Each flag gets a default so the suite runs sensibly with zero configuration for local development. A plain fixture then reads the resolved value via request.config.getoption("--base-url"), and any OTHER fixture (here, credentials) can depend on that fixture the normal pytest way, so environment-dependent values (which login credentials belong to which environment) cascade automatically without every test needing to reason about --env directly.

Worked example

# conftest.py
import pytest

def pytest_addoption(parser):
    parser.addoption("--base-url", action="store", default="https://staging.example.com")
    parser.addoption("--env", action="store", default="staging")

@pytest.fixture
def base_url(request):
    return request.config.getoption("--base-url")

@pytest.fixture
def env(request):
    return request.config.getoption("--env")

@pytest.fixture
def credentials(env):
    creds = {"staging": ("staginguser", "stagingpass"), "prod": ("produser", "prodpass")}
    return creds.get(env, creds["staging"])

# test_config.py
def test_config_reflects_cli_overrides(base_url, env, credentials):
    print(f"base_url={base_url} env={env} credentials={credentials}")
    expected_creds = {"staging": ("staginguser", "stagingpass"), "prod": ("produser", "prodpass")}
    assert credentials == expected_creds[env]
    assert base_url.startswith("https://")

Run with no flags (defaults):

$ pytest test_config.py -v -s
base_url=https://staging.example.com env=staging credentials=('staginguser', 'stagingpass')
PASSED

Run with the CLI overrides:

$ pytest test_config.py -v -s --base-url=https://prod.example.com --env=prod
base_url=https://prod.example.com env=prod credentials=('produser', 'prodpass')
PASSED

The same test passes in both cases, correctly reflecting whichever environment was requested, which confirms the override mechanism actually reaches the test rather than being silently ignored.

Trade-offs and pitfalls

A common mistake is reading os.environ directly inside individual tests instead of centralizing configuration through fixtures like this: it works for one test, but scatters environment-reading logic across the suite and makes it much harder to see, in one place, everything the suite's behavior depends on. A second pitfall is forgetting that pytest_addoption must live in a conftest.py that pytest actually discovers (typically at or above the test's directory); registering the same option name in two different conftest files pytest collects into the same run raises a clear "option already added" error at collection time, which is a good early signal rather than a silent conflict.

Direct answer

Open the login URL, wait explicitly for the username and password fields to exist before touching them, fill and submit the form, then wait for the resulting page's title to reflect the post-login state before asserting on it, so the test never races the page's own load or redirect timing.

Structured elaboration

The key discipline this exercise tests is using an EXPLICIT wait (WebDriverWait with an expected condition) at every point where the DOM state genuinely depends on timing: once for the username field appearing after the login page loads, and once for the title changing after the redirect following a successful submit. A test that instead used time.sleep(2) in both places would usually pass locally and become flaky in CI the moment the page is slower than the guessed sleep duration; an explicit wait polls until the condition is true (or times out), so it is correct regardless of how fast or slow the page actually is.

Worked example

import pytest
from selenium.webdriver.common.by import By
from selenium.webdriver.support.ui import WebDriverWait
from selenium.webdriver.support import expected_conditions as EC

def do_login(driver, base_url="https://example.com/login"):
    driver.get(base_url)
    wait = WebDriverWait(driver, 10)
    wait.until(EC.presence_of_element_located((By.ID, "username"))).send_keys("testuser")
    driver.find_element(By.ID, "password").send_keys("P@ssw0rd")
    driver.find_element(By.ID, "login-btn").click()
    return wait.until(EC.title_contains("Dashboard"))

@pytest.fixture
def driver():
    from selenium import webdriver
    d = webdriver.Chrome()   # real suite: chromedriver on PATH, as the question assumes
    yield d
    d.quit()

def test_login_direct(driver):
    assert do_login(driver) is True

No browser binary is available in this environment, so the control flow (correct locator usage, correct wait-then-act ordering, correct final assertion) was verified separately against a mocked driver (a realistic title string and a stub find_element) instead of webdriver.Chrome():

from unittest.mock import MagicMock, call

@pytest.fixture
def driver():
    d = MagicMock()
    d.title = "My App - Dashboard"
    d.find_element.return_value = MagicMock()
    return d

def test_login_direct_mocked(driver):
    result = do_login(driver)
    assert result is True
    driver.get.assert_called_once_with("https://example.com/login")
    assert driver.find_element.call_args_list == [
        call(By.ID, "username"),
        call(By.ID, "password"),
        call(By.ID, "login-btn"),
    ]

$ pytest test_direct_login_mocked.py -v
test_login_direct_mocked PASSED
1 passed in 0.11s

This confirms driver.get was called with the right URL exactly once, and find_element was called for username (via the WebDriverWait-guarded lookup), password, and login-btn, in the order the function performs them.

Trade-offs and pitfalls

The most common mistake on this exact exercise is waiting only ONCE, for the username field, and then asserting on the title immediately without a second wait: since the redirect to the dashboard is itself asynchronous, the title assertion can fire before the navigation completes and fail intermittently depending on network/render speed, which is exactly the kind of test that passes on a fast machine and flakes in CI. A second pitfall is hardcoding the 10-second timeout inline on every wait call rather than as one named constant, which becomes a maintenance problem the moment a team decides to tune it suite-wide.

Direct answer

setUp runs before test_login_status and builds self.user, the fixture; the test body calls login(self.user) (the action) and then self.assertEqual(status, 200) (the verification). If the assertion fails, unittest raises an AssertionError internally, the test framework catches it, marks that test method as FAILED, records the traceback showing exactly which values did not match, and continues on to any other test methods rather than crashing the whole run.

Structured elaboration

Every unittest.TestCase method follows the same three-part shape this snippet demonstrates:

• Fixture: setUp runs immediately before EVERY test method in the class (not once per class), so if there were a second test method here it would get its own fresh self.user dict, not a shared/mutated one from a previous test.
• Action: the line that exercises the system under test, here login(self.user), whose return value is captured for inspection.
• Verification: the self.assertEqual(...) call, which is where the test actually decides pass or fail; everything before it is just getting into position to make that judgment.

On what happens when the assertion fails: unittest's assert methods (assertEqual, assertTrue, etc.) raise AssertionError when the condition does not hold. The test RUNNER (whether unittest's own runner, or pytest running a unittest.TestCase) catches that exception per-test, records a FAILED result with the full traceback and the specific values involved, and moves on to the next test rather than letting one failure abort the whole suite, exactly the same isolation guarantee a well-written pytest suite gives you.

Worked example

import unittest

def login(user):
    if user['password'] == 'pass':
        return 200
    return 401

class TestLogin(unittest.TestCase):
    def setUp(self):
        self.user = {'username': 'alice', 'password': 'pass'}

    def test_login_status(self):
        status = login(self.user)
        self.assertEqual(status, 200)

if __name__ == "__main__":
    unittest.main(verbosity=2)

Running the snippet as given (correct password):

test_login_status (__main__.TestLogin.test_login_status) ... ok

Ran 1 test in 0.000s

OK

Now the same shape with a wrong password, to show exactly what a failure looks like:

class TestLoginFails(unittest.TestCase):
    def setUp(self):
        self.user = {'username': 'alice', 'password': 'WRONG'}

    def test_login_status(self):
        status = login(self.user)
        self.assertEqual(status, 200)

test_login_status (__main__.TestLoginFails.test_login_status) ... FAIL

======================================================================
FAIL: test_login_status (__main__.TestLoginFails.test_login_status)
----------------------------------------------------------------------
Traceback (most recent call last):
  ...
    self.assertEqual(status, 200)
AssertionError: 401 != 200

Ran 1 test in 0.000s

FAILED (failures=1)

The failure is reported as its own line naming the exact expected-vs-actual values (401 != 200), and the run still completes and reports a final tally rather than crashing.

Trade-offs and pitfalls

A common miscue is describing setUp as running "once for the whole class"; it actually runs once per TEST METHOD, which matters the moment a class has more than one test and one of them mutates self.user (the next test still starts from a clean dict, not the mutated one, precisely because setUp reran). Another pitfall is conflating an assertion failure with an unexpected exception: if login itself raised, say, a KeyError because user was missing the 'password' key, unittest reports that as an ERROR, not a FAILURE, a distinction that matters when triaging a big test run because errors usually point at a broken test or environment, while failures usually point at the system under test actually behaving differently than expected.

Direct answer

Compare the CI environment to the developer machine along every axis that could plausibly differ (browser/driver version, screen resolution, network latency to dependencies, CPU/parallelism load, timezone/locale), then run two targeted experiments that isolate WHICH axis actually matters rather than guessing: reproduce the CI conditions locally, and add CI-only artifact capture so the next failure is diagnosable without needing to reproduce it live.

Structured elaboration

Checklist of things to inspect: browser and driver versions (CI images often auto-update independently of a developer's pinned local version); headless versus headed mode (headless Chrome has historically had subtly different rendering/timing characteristics than headed); screen/viewport resolution (a CI container's default resolution is often smaller than a developer's monitor, which can change what is "visible" and therefore clickable); CPU and memory constraints (CI runners are frequently smaller/shared machines, so anything timing-sensitive is more likely to lose a race there than on a fast developer laptop); parallel execution (CI often runs many tests concurrently, competing for the same resources a developer's single local run never contends for); and environment-specific config (a different base URL, feature flags, or seeded test data than what is on a developer's local environment).

Two concrete experiments:

1. Run the SAME test, in headless mode, at the CI container's actual screen resolution, ALONE (not in parallel with other tests) on the local machine. If it still passes locally under those conditions, the CI-only failure is more likely about resource contention (CPU/parallelism) than about headless mode or resolution specifically; if it now fails locally too, you have isolated the cause to headless mode or resolution without ever touching the real CI infrastructure.
2. Run the SAME test locally under artificial CPU throttling and forced parallel load (starting several other CPU-heavy processes, or using an OS-level CPU limiter) to simulate the CI runner's resource constraints. If the failure reproduces under load but not otherwise, that strongly implicates contention/timing rather than a genuine environment configuration difference.

Artifacts to capture on every CI failure going forward (so a NEXT failure does not require this same investigation from scratch): a screenshot at the moment of failure, the browser's console/network logs, the exact browser and driver version strings, and the CI runner's resource state (CPU/memory) at failure time.

Trade-offs and pitfalls

The most common mistake is jumping straight to "just add a longer wait" without running either experiment, which sometimes masks the symptom on the exact CI hardware tested today while leaving the actual root cause (a genuine resource-contention or configuration difference) unaddressed and likely to resurface the next time CI load patterns shift. A second pitfall is testing only ONE variable at a time forever: if headless-mode-alone and resolution-alone both pass locally but the combination (headless AND CI resolution AND parallel load) still only fails in real CI, the interaction between multiple factors, not any single one, is the actual cause, which is worth checking explicitly rather than concluding "environment is identical, must be a flaky test" prematurely.