Test Automation Scripting Questions

Direct answer

@pytest.mark.parametrize runs the same test body once per row of data you supply, each row becoming its own reported test case with its own pass/fail and its own readable ID in the output, instead of one test that loops internally and reports a single result for all three credential pairs.

Structured elaboration

The pattern has three parts: a parameter list (here, three (username, password, should_succeed) tuples), a test function whose parameters are named exactly to match, and a fixture or setup for anything shared across all rows (here, the object you log in against). The decorator generates one independent test invocation per tuple, so a failure on the invalid-credentials row is reported as test_login[invalid-wrong-False] failing while the two valid-credential rows still show as passed, which is the entire point: a candidate who instead writes for creds in credential_list: assert login(...) gets one pass/fail for the whole loop and loses per-case visibility the moment more than one row is involved.

Worked example

import pytest

class FakeAuthAPI:
    """Stand-in for a real login endpoint/UI so the mechanic can be shown without a server or browser."""
    _valid = {("user1", "pass1"), ("user2", "pass2")}
    def login(self, username, password):
        return (username, password) in self._valid

@pytest.fixture
def api():
    return FakeAuthAPI()

@pytest.mark.parametrize("username, password, should_succeed", [
    ("user1", "pass1", True),
    ("user2", "pass2", True),
    ("invalid", "wrong", False),
])
def test_login(api, username, password, should_succeed):
    result = api.login(username, password)
    assert result is should_succeed

Running it:

$ pytest test_login_parametrize.py -v
collected 3 items

test_login_parametrize.py::test_login[user1-pass1-True] PASSED
test_login_parametrize.py::test_login[user2-pass2-True] PASSED
test_login_parametrize.py::test_login[invalid-wrong-False] PASSED

3 passed in 0.01s

Three independent results, one per credential pair, each individually addressable in a CI report.

The same mechanic works on a trivial pure function too, which is worth knowing as an even more compact way to demonstrate the syntax when the interviewer only wants to see the API, not a login flow:

import pytest

def is_palindrome(s):
    return s == s[::-1]

@pytest.mark.parametrize("s, expected", [
    ("radar", True),
    ("hello", False),
    ("", True),
])
def test_is_palindrome(s, expected):
    assert is_palindrome(s) == expected

$ pytest -v
test_is_palindrome[radar-True] PASSED
test_is_palindrome[hello-False] PASSED
test_is_palindrome[-True] PASSED
3 passed in 0.01s

Trade-offs and pitfalls

The most common mistake is parametrizing a fixture instead of the test when the setup itself needs to vary (use @pytest.fixture(params=[...]) for that, not @pytest.mark.parametrize on the test), which produces confusing double-parametrization if both are used for the same axis. A second pitfall is mutable default state leaking across parametrized runs: if the fixture in this example returned a module-level object instead of a fresh one per test, a mutation in one row (e.g. banning user1 after a failed attempt) could silently affect a later row's result depending on execution order. Finally, once the list of tuples grows past a handful of rows, hardcoding them inline stops scaling. That is the point where the case for pulling data from an external file becomes a genuine data-management decision (how it is stored, generated, and kept in sync with the tests) that goes beyond this one script.

Direct answer

The weakness is that the test asserts an immediate count delta (len(after) == len(before) + 1) right after the POST returns, with no allowance for the possibility that the created user is not yet visible in a subsequent GET; if the system has any eventual-consistency lag between accepting the write and reflecting it in reads (an async index, a read replica, a cache), this test can fail even when the system is working exactly as designed, or worse, can PASS for the wrong reason if some unrelated user happened to be created by another test at the same moment. Fix it by polling for the SPECIFIC created resource to appear, with a bounded timeout, instead of asserting an instantaneous count.

Structured elaboration

Two separate problems live in that one assertion. First, timing: api_client.post(...) returning does not guarantee the write is immediately visible to a subsequent GET; many real systems accept a write synchronously but propagate it to whatever the read path queries (a search index, a cache, a read replica) asynchronously, which is a completely normal and correct design, not a bug, so a test that assumes synchronous visibility is testing an assumption the system never promised. Second, specificity: counting total users before and after is fragile even ignoring timing, since it says nothing about WHICH user appeared; if the count happens to be off by one for a completely unrelated reason (test isolation failure, a concurrent test running against the same environment), the assertion could pass or fail for reasons that have nothing to do with whether bob@example.com specifically was created.

The fix addresses both: poll the GET endpoint for the SPECIFIC email just created (not just any count increase) for a bounded time, and only fail if it never appears within that window, which correctly tolerates real, bounded eventual-consistency lag while still catching a genuine failure to create the user at all.

Worked example

import time

def test_create_user_robust(api_client, target_email='bob@example.com', timeout=1.0, poll=0.02):
    before = api_client.get('/users').json()
    api_client.post('/users', json={'email': target_email})
    deadline = time.time() + timeout
    while time.time() < deadline:
        after = api_client.get('/users').json()
        if any(u.get('email') == target_email for u in after):
            assert len(after) >= len(before) + 1
            return True
        time.sleep(poll)
    raise AssertionError(f"{target_email} never became visible within {timeout}s")


import threading

class FakeAPIClient:
    """Simulates a system with a 0.05s asynchronous write-visibility delay: POST is
    accepted immediately, but the new record only appears in GET results after a short
    async propagation delay (like an async index or read replica)."""
    def __init__(self, delay=0.05):
        self.delay = delay
        self._visible = []
        self._lock = threading.Lock()

    def get(self, path):
        class R:
            def __init__(self, data): self._data = data
            def json(self): return self._data
        with self._lock:
            return R(list(self._visible))

    def post(self, path, json):
        def make_visible():
            time.sleep(self.delay)
            with self._lock:
                self._visible.append(json)
        threading.Thread(target=make_visible).start()

def weak_test(api_client, target_email):
    before = api_client.get('/users').json()
    api_client.post('/users', json={'email': target_email})
    after = api_client.get('/users').json()
    assert len(after) == len(before) + 1

N = 5
weak_failures = 0
for i in range(N):
    client = FakeAPIClient(delay=0.05)
    try:
        weak_test(client, target_email=f"bob{i}@example.com")
    except AssertionError:
        weak_failures += 1
    time.sleep(0.1)  # let the background thread finish before the next run

robust_failures = 0
for i in range(N):
    client = FakeAPIClient(delay=0.05)
    try:
        test_create_user_robust(client, target_email=f"bob{i}@example.com")
    except AssertionError:
        robust_failures += 1

print(f"weak (count-based, no wait) assertion failed {weak_failures}/{N} runs due to the visibility race")
print(f"robust (poll-for-visibility) assertion failed {robust_failures}/{N} runs")

Running it:

weak (count-based, no wait) assertion failed 5/5 runs due to the visibility race
robust (poll-for-visibility) assertion failed 0/5 runs

The original weak assertion failed on every single run because it never allows for the visibility lag; the fixed version, polling for the specific user with a timeout, passed on every run.

Trade-offs and pitfalls

The most common mistake in "fixing" this is simply adding a fixed time.sleep(1) before the GET, which trades a fast, flaky test for a slow, still-technically-flaky test (it works until the real lag exceeds whatever sleep duration was guessed, and it wastes a full second on every run even when the system responds in milliseconds); polling with a bounded timeout gets the correctness of waiting without paying the worst-case delay every single time. A second pitfall is polling forever with no timeout at all, which turns a genuine backend outage (the user is NEVER created) into a hung test rather than a clear, timely failure; the timeout is what preserves the test's ability to catch a real bug instead of just tolerating a slow one.

Direct answer

Configure headless mode, a custom download directory, disabled extensions, and a proxy through the browser-specific Options class (ChromeOptions/FirefoxOptions), passed into the driver constructor, rather than the older DesiredCapabilities dictionary-style API, which Selenium 4 has moved away from in favor of typed, browser-specific options objects.

Structured elaboration

Each setting maps to a specific Options method or preference, and the exact mechanism differs between Chrome and Firefox even though the concept is the same on both:

Chrome (ChromeOptions): headless mode is options.add_argument('--headless=new') for current Chrome (the =new headless implementation is closer to real headed Chrome than the legacy headless mode); a custom download directory and disabling the "always ask where to save" prompt are Chrome preferences set via options.add_experimental_option('prefs', {...}); disabling extensions is options.add_argument('--disable-extensions'); a proxy is configured via Selenium's Proxy class or a direct --proxy-server= argument.

Firefox (FirefoxOptions): the mechanism is preference-based rather than the Chrome prefs-experimental-option pattern. Headless mode is options.add_argument('-headless') (single dash, not Chrome's double dash); a custom download directory is a trio of set_preference calls (browser.download.folderList set to 2 for "custom location," browser.download.dir set to the path, and browser.helperApps.neverAsk.saveToDisk set to the MIME type(s) that should download without a prompt); there is no Chrome-style --disable-extensions flag for Firefox, since a WebDriver-launched Firefox session already starts from a fresh, empty profile with no extensions installed, so "disabling extensions" in Firefox is mostly about making sure none get auto-installed by policy, which extensions.autoDisableScopes set to 0 addresses; a proxy is a set of network.proxy.* preferences (network.proxy.type = 1 for manual, plus network.proxy.http/network.proxy.http_port/network.proxy.ssl/network.proxy.ssl_port) rather than a single command-line flag.

DesiredCapabilities was the pre-Selenium-4 way to configure ALL of this, for both browsers: a loosely-typed dictionary of capability names and values passed to the driver, with no browser-specific validation until the browser driver itself rejected something it did not understand. The modern Options/BrowserOptions classes (ChromeOptions, FirefoxOptions, EdgeOptions) are typed, browser-specific, and validated by the binding itself, catching a misspelled or unsupported option earlier (at Python-object-construction or driver-instantiation time) rather than as an opaque server-side rejection.

Worked example

from selenium.webdriver.chrome.options import Options

def build_chrome_options(download_dir, proxy=None):
    options = Options()
    options.add_argument('--headless=new')
    options.add_argument('--disable-extensions')
    options.add_experimental_option('prefs', {
        'download.default_directory': download_dir,
        'download.prompt_for_download': False,
    })
    if proxy:
        options.add_argument(f'--proxy-server={proxy}')
    return options

# usage: driver = webdriver.Chrome(options=build_chrome_options('/tmp/downloads'))

from selenium.webdriver.firefox.options import Options as FirefoxOptions

def build_firefox_options(download_dir, proxy=None):
    options = FirefoxOptions()
    options.add_argument('-headless')
    options.set_preference('browser.download.folderList', 2)
    options.set_preference('browser.download.dir', download_dir)
    options.set_preference('browser.helperApps.neverAsk.saveToDisk', 'application/octet-stream')
    options.set_preference('extensions.autoDisableScopes', 0)
    if proxy:
        host, port = proxy.split(':')
        options.set_preference('network.proxy.type', 1)
        options.set_preference('network.proxy.http', host)
        options.set_preference('network.proxy.http_port', int(port))
        options.set_preference('network.proxy.ssl', host)
        options.set_preference('network.proxy.ssl_port', int(port))
    return options

# usage: driver = webdriver.Firefox(options=build_firefox_options('/tmp/downloads'))

Both functions were run directly against the installed Selenium 4 package (no browser needed to build an Options object): build_chrome_options('/tmp/downloads', proxy='127.0.0.1:8080') produces arguments = ['--headless=new', '--disable-extensions', '--proxy-server=127.0.0.1:8080'] and experimental_options = {'prefs': {'download.default_directory': '/tmp/downloads', 'download.prompt_for_download': False}}; build_firefox_options('/tmp/downloads', proxy='127.0.0.1:8080') produces arguments = ['-headless'] and a preferences dict containing every browser.download.*, extensions.autoDisableScopes, and network.proxy.* key set above. Both Options objects construct without error, confirming the option/preference names are accepted by the current Selenium binding.

This uses Selenium 4's typed Options object exclusively for both browsers; the legacy equivalent would instead build a DesiredCapabilities.CHROME.copy() (or .FIREFOX.copy()) dictionary and merge these settings into it by hand, with no validation until the browser session actually started.

Trade-offs and pitfalls

The most common mistake is mixing DesiredCapabilities and Options in the same codebase inconsistently across a suite (a legacy pattern some teams never fully migrated away from), which makes configuration harder to reason about since two different mechanisms can both be setting overlapping options; standardizing entirely on Options for new code, and migrating the rest opportunistically, avoids that confusion. A second pitfall specific to headless mode: legacy Chrome headless (--headless without =new) had real, documented behavioral differences from headed Chrome (different default window size, some rendering differences), which historically caused tests that passed headed to fail headless or vice versa; the newer --headless=new implementation closes most of that gap but it is still worth verifying a suite behaves identically in both modes before trusting headless CI runs as equivalent to local headed development. A third, Firefox-specific pitfall: assuming Chrome's prefs/experimental-option pattern applies to Firefox will fail silently or raise, since Firefox configuration is set_preference calls, not a single dict passed as an experimental option; porting a Chrome options-builder to Firefox by renaming the class alone (without switching the mechanism) is a common copy-paste bug.

Direct answer

Write the test to CONSUME an existing LoginPage object: the page object owns the locators and the one login(username, password) action, and the test itself only calls that action and asserts on the outcome, with setup/teardown and parametrized credential sets handled at the test level, not inside the page object.

Structured elaboration

A Page Object is a thin wrapper around ONE page's elements and the actions a user can take on it. The dividing line that matters here: the page object exposes a business-meaningful method (login), never raw selectors leaking into the test, and it does not assert anything itself, since assertions belong to the test, not the page. Parameterizing the credential sets is a separate, orthogonal concern from the page object's design: @pytest.mark.parametrize supplies each credential pair to the same test body, and the page object does not need to know or care how many times it gets called.

For setup/teardown, a real test would create a fresh WebDriver session per test (function-scoped fixture, driver.quit() in teardown) so credential sets do not share browser/session state across parametrized runs, which matters because a leftover cookie or "remember me" state from one credential pair could otherwise mask a login bug on the next.

Worked example

import pytest
from selenium.webdriver.common.by import By
from selenium.webdriver.support.ui import WebDriverWait
from selenium.webdriver.support import expected_conditions as EC

class LoginPage:
    USERNAME = (By.ID, "username")
    PASSWORD = (By.ID, "password")
    SUBMIT = (By.ID, "login-btn")

    def __init__(self, driver):
        self.driver = driver
        self.wait = WebDriverWait(driver, 10)

    def login(self, username, password):
        self.wait.until(EC.presence_of_element_located(self.USERNAME)).send_keys(username)
        self.driver.find_element(*self.PASSWORD).send_keys(password)
        self.driver.find_element(*self.SUBMIT).click()

@pytest.fixture
def driver():
    from selenium import webdriver
    d = webdriver.Chrome()          # a real suite: one fresh session per test
    yield d
    d.quit()

@pytest.mark.parametrize("username,password", [
    ("user1", "pass1"),
    ("user2", "pass2"),
    ("invalid", "wrongpass"),
])
def test_login_flow(driver, username, password):
    driver.get("https://example.com/login")
    page = LoginPage(driver)
    page.login(username, password)

No browser binary is available in this environment, so the page object's CONTROL FLOW (not real browser rendering) was verified separately against a mocked driver, asserting the exact sequence of locator calls LoginPage.login makes for each parametrized credential pair:

from unittest.mock import MagicMock

@pytest.fixture
def driver():
    d = MagicMock()
    d.find_element.return_value = MagicMock()
    return d

@pytest.mark.parametrize("username,password", [
    ("user1", "pass1"),
    ("user2", "pass2"),
    ("invalid", "wrongpass"),
])
def test_login_flow_mocked(driver, username, password):
    page = LoginPage(driver)
    page.login(username, password)
    assert driver.find_element.call_args_list == [
        ((By.ID, "username"),),
        ((By.ID, "password"),),
        ((By.ID, "login-btn"),),
    ]

$ pytest test_login_pom_mock.py -v
test_login_flow_mocked[user1-pass1] PASSED
test_login_flow_mocked[user2-pass2] PASSED
test_login_flow_mocked[invalid-wrongpass] PASSED
3 passed in 0.16s

This confirms, against the real installed Selenium 4 API (By, WebDriverWait, expected_conditions, not hand-typed from memory), that the page object calls find_element(By.ID, "username") (via the WebDriverWait-guarded lookup), find_element(By.ID, "password"), and find_element(By.ID, "login-btn") for every credential pair, in that order, regardless of which pair is under test. (The webdriver.Chrome()-based driver fixture above is what a real suite would run against a live browser; this mocked version is only for verifying the page object's logic here.)

Trade-offs and pitfalls

The most common mistake is putting the assertion inside the page object (e.g. a login_succeeded() method that itself asserts), which makes the page object opinionated about what "success" means for every caller; a page object should expose state (is the dashboard visible?) and let the TEST decide what that state should be for a given credential pair. A second pitfall is sharing one browser session across all three parametrized credential sets to save setup time: that shortcut trades correctness for speed, since a stale session can hide or fake a login result. Finally, WebDriverWait with a hardcoded 10-second timeout is a reasonable default but should be a configurable constant, not repeated as a magic number across every page object in a real suite.

Direct answer

Capture the browser's console log and network activity during the test run using Selenium 4's Chrome DevTools Protocol (CDP) integration, or the simpler driver.get_log('browser') API for console-only capture, and attach both to the CI failure report so a teammate can see the client-side JavaScript error or failed network call without ever needing to reproduce the flaky run locally.

Structured elaboration

Two complementary capture mechanisms exist, at different levels of depth. driver.get_log('browser') is the lighter-weight option: it returns the browser's own console log entries (JavaScript errors, warnings, console.log output) as a simple list, with no setup beyond enabling browser logging in the driver's capabilities. Selenium 4's native CDP integration (driver.execute_cdp_cmd(...), or the higher-level driver.get_log/Network domain listeners in newer bindings) goes further, giving access to the full Chrome DevTools Network domain, request/response headers, timing, and failed requests, which is what you need when the suspected cause is a failed or slow NETWORK call rather than a pure JavaScript error.

The most useful log types for this scenario specifically: console logs (to catch a JavaScript error that prevented an event listener from attaching, matching this question's premise), and network/CDP Network.responseReceived/Network.loadingFailed events (to catch a background request that failed or never completed, which can silently break functionality that depended on its result without ever showing up as a visible error). Attaching both to the CI report, alongside the existing screenshot-on-failure artifact, means a teammate reading the failure the next morning has the same diagnostic picture you would have had watching it fail live.

Worked example

def capture_browser_diagnostics(driver):
    console_logs = driver.get_log('browser')
    driver.execute_cdp_cmd('Network.enable', {})
    # In a real run, CDP network events are captured via a listener registered before
    # navigation; a fuller CDP-based capture pipeline records request/response pairs as
    # they occur rather than fetching them after the fact.
    return {
        "console": console_logs,
        # network events would be appended here by the listener during the test
    }

Attaching this to the CI report (as a JSON artifact alongside the existing screenshot) means a JavaScript error or a failed background request is visible without needing to reproduce the failure interactively.

Trade-offs and pitfalls

The most common mistake is capturing ONLY the console log and assuming that covers "client-side issues": a failed background network call that the application handles silently (retries without logging, or simply drops the result) produces NO console error at all, so relying on console logs alone would miss exactly the class of bug this question's premise describes. A second pitfall is enabling verbose CDP network capture on every single test run regardless of pass/fail, which adds real overhead at scale; a common middle ground is capturing lightweight console logs always, and only spinning up the heavier CDP network capture path when a test actually fails, similar to how screenshot capture is usually failure-triggered rather than always-on.