Test Automation Scripting Questions

Direct answer

setUp runs before test_login_status and builds self.user, the fixture; the test body calls login(self.user) (the action) and then self.assertEqual(status, 200) (the verification). If the assertion fails, unittest raises an AssertionError internally, the test framework catches it, marks that test method as FAILED, records the traceback showing exactly which values did not match, and continues on to any other test methods rather than crashing the whole run.

Structured elaboration

Every unittest.TestCase method follows the same three-part shape this snippet demonstrates:

• Fixture: setUp runs immediately before EVERY test method in the class (not once per class), so if there were a second test method here it would get its own fresh self.user dict, not a shared/mutated one from a previous test.
• Action: the line that exercises the system under test, here login(self.user), whose return value is captured for inspection.
• Verification: the self.assertEqual(...) call, which is where the test actually decides pass or fail; everything before it is just getting into position to make that judgment.

On what happens when the assertion fails: unittest's assert methods (assertEqual, assertTrue, etc.) raise AssertionError when the condition does not hold. The test RUNNER (whether unittest's own runner, or pytest running a unittest.TestCase) catches that exception per-test, records a FAILED result with the full traceback and the specific values involved, and moves on to the next test rather than letting one failure abort the whole suite, exactly the same isolation guarantee a well-written pytest suite gives you.

Worked example

import unittest

def login(user):
    if user['password'] == 'pass':
        return 200
    return 401

class TestLogin(unittest.TestCase):
    def setUp(self):
        self.user = {'username': 'alice', 'password': 'pass'}

    def test_login_status(self):
        status = login(self.user)
        self.assertEqual(status, 200)

if __name__ == "__main__":
    unittest.main(verbosity=2)

Running the snippet as given (correct password):

test_login_status (__main__.TestLogin.test_login_status) ... ok

Ran 1 test in 0.000s

OK

Now the same shape with a wrong password, to show exactly what a failure looks like:

class TestLoginFails(unittest.TestCase):
    def setUp(self):
        self.user = {'username': 'alice', 'password': 'WRONG'}

    def test_login_status(self):
        status = login(self.user)
        self.assertEqual(status, 200)

test_login_status (__main__.TestLoginFails.test_login_status) ... FAIL

======================================================================
FAIL: test_login_status (__main__.TestLoginFails.test_login_status)
----------------------------------------------------------------------
Traceback (most recent call last):
  ...
    self.assertEqual(status, 200)
AssertionError: 401 != 200

Ran 1 test in 0.000s

FAILED (failures=1)

The failure is reported as its own line naming the exact expected-vs-actual values (401 != 200), and the run still completes and reports a final tally rather than crashing.

Trade-offs and pitfalls

A common miscue is describing setUp as running "once for the whole class"; it actually runs once per TEST METHOD, which matters the moment a class has more than one test and one of them mutates self.user (the next test still starts from a clean dict, not the mutated one, precisely because setUp reran). Another pitfall is conflating an assertion failure with an unexpected exception: if login itself raised, say, a KeyError because user was missing the 'password' key, unittest reports that as an ERROR, not a FAILURE, a distinction that matters when triaging a big test run because errors usually point at a broken test or environment, while failures usually point at the system under test actually behaving differently than expected.

Direct answer

For uploads, send the file path directly to the <input type="file"> element with send_keys rather than trying to automate the native OS file-picker dialog; for downloads, configure the browser to save to a known, per-test directory with prompts disabled, then poll that directory until the expected file exists AND is no longer a partial in-progress download, before verifying its contents and cleaning it up.

Structured elaboration

Uploads: WebDriver cannot interact with a native OS dialog at all, since it lives outside the browser's DOM entirely; the reliable approach sidesteps the dialog completely by finding the underlying <input type="file"> element (even if it is visually hidden behind a styled button) and calling send_keys(absolute_file_path) on it directly, which the browser treats exactly as if a user had picked that file. If no such input exists (a genuinely native-only upload widget), that is one of the rare cases that legitimately needs OS-level tooling, and is worth flagging as an edge case rather than the default plan.

Downloads, across browsers and CI runners, need three coordinated pieces: (1) a per-test or per-run download directory, configured via browser options (prefs in Chrome, browser.download.dir in Firefox) rather than the OS default Downloads folder, so parallel CI jobs never collide on the same files; (2) prompts disabled (profile.default_content_settings.popups: 0 and the "always ask where to save" preference turned off) so the download starts immediately without a dialog WebDriver cannot see; (3) verification that polls for the FINAL filename to exist while the browser's in-progress marker (Chrome's .crdownload, Firefox's .part) is gone, not just checking for the final filename's existence the instant the download starts, since large files spend real time in the partial state.

Headless mode needs one extra step in older Chrome versions: enabling downloads in headless mode required an explicit CDP command (Page.setDownloadBehavior) because headless Chrome historically disabled downloads by default for security reasons; modern Chrome/Selenium handles this more transparently, but it is worth knowing as a historical gotcha if an older pinned Chrome version is in play.

Worked example

import os, time, tempfile, threading

def wait_for_download(download_dir, filename, timeout=10, poll=0.2):
    deadline = time.time() + timeout
    target = os.path.join(download_dir, filename)
    partial = target + ".crdownload"
    while time.time() < deadline:
        if os.path.exists(target) and not os.path.exists(partial):
            return target
        time.sleep(poll)
    raise TimeoutError(f"{filename} did not finish downloading within {timeout}s")


download_dir = tempfile.mkdtemp()
filename = "report.csv"
partial_path = os.path.join(download_dir, filename + ".crdownload")
final_path = os.path.join(download_dir, filename)

# Simulate a browser: write the partial marker immediately, then finish the
# download after a short delay by writing the final file and removing the
# partial marker, on a background thread.
with open(partial_path, "w") as f:
    f.write("")

def finish_download():
    time.sleep(0.5)
    with open(final_path, "w") as f:
        f.write("id,amount\n1,42\n")
    os.remove(partial_path)

threading.Thread(target=finish_download).start()

result_path = wait_for_download(download_dir, filename, timeout=5, poll=0.1)
print(f"download completed at: {result_path} | exists: {os.path.exists(result_path)} | partial gone: {not os.path.exists(partial_path)}")

Running it:

download completed at: /tmp/.../report.csv | exists: True | partial gone: True

The poll correctly waited past the partial-file window rather than returning the instant the (still-downloading) target check might have raced a browser that writes the final filename before the content is fully flushed.

Trade-offs and pitfalls

The most common upload mistake is trying to automate the native file-picker dialog with OS-level tooling by default, when the send_keys-to-hidden-input approach works for the overwhelming majority of real upload widgets and is far more reliable in headless CI (native dialogs frequently cannot be driven at all in a headless container). The most common download mistake is checking only for the final filename's existence without also checking that the partial-download marker is gone, which produces a race where the test reads a file that is still being written and gets a truncated or empty result. Finally, per-test download directories (not a single shared Downloads folder) are essential for parallel CI; without that isolation, two tests downloading files with the same name can silently overwrite or read each other's downloads.

Direct answer

Compare the CI environment to the developer machine along every axis that could plausibly differ (browser/driver version, screen resolution, network latency to dependencies, CPU/parallelism load, timezone/locale), then run two targeted experiments that isolate WHICH axis actually matters rather than guessing: reproduce the CI conditions locally, and add CI-only artifact capture so the next failure is diagnosable without needing to reproduce it live.

Structured elaboration

Checklist of things to inspect: browser and driver versions (CI images often auto-update independently of a developer's pinned local version); headless versus headed mode (headless Chrome has historically had subtly different rendering/timing characteristics than headed); screen/viewport resolution (a CI container's default resolution is often smaller than a developer's monitor, which can change what is "visible" and therefore clickable); CPU and memory constraints (CI runners are frequently smaller/shared machines, so anything timing-sensitive is more likely to lose a race there than on a fast developer laptop); parallel execution (CI often runs many tests concurrently, competing for the same resources a developer's single local run never contends for); and environment-specific config (a different base URL, feature flags, or seeded test data than what is on a developer's local environment).

Two concrete experiments:

1. Run the SAME test, in headless mode, at the CI container's actual screen resolution, ALONE (not in parallel with other tests) on the local machine. If it still passes locally under those conditions, the CI-only failure is more likely about resource contention (CPU/parallelism) than about headless mode or resolution specifically; if it now fails locally too, you have isolated the cause to headless mode or resolution without ever touching the real CI infrastructure.
2. Run the SAME test locally under artificial CPU throttling and forced parallel load (starting several other CPU-heavy processes, or using an OS-level CPU limiter) to simulate the CI runner's resource constraints. If the failure reproduces under load but not otherwise, that strongly implicates contention/timing rather than a genuine environment configuration difference.

Artifacts to capture on every CI failure going forward (so a NEXT failure does not require this same investigation from scratch): a screenshot at the moment of failure, the browser's console/network logs, the exact browser and driver version strings, and the CI runner's resource state (CPU/memory) at failure time.

Trade-offs and pitfalls

The most common mistake is jumping straight to "just add a longer wait" without running either experiment, which sometimes masks the symptom on the exact CI hardware tested today while leaving the actual root cause (a genuine resource-contention or configuration difference) unaddressed and likely to resurface the next time CI load patterns shift. A second pitfall is testing only ONE variable at a time forever: if headless-mode-alone and resolution-alone both pass locally but the combination (headless AND CI resolution AND parallel load) still only fails in real CI, the interaction between multiple factors, not any single one, is the actual cause, which is worth checking explicitly rather than concluding "environment is identical, must be a flaky test" prematurely.

Direct answer

Wrap the "does the table contain this row" check inside an explicit wait that polls until the table has more than zero rows, then search the loaded rows for the target value, structured behind a small page-object-style class for the table so the waiting and searching logic lives in one place rather than being copy-pasted into every test that touches this table.

Structured elaboration

AJAX-loaded content is exactly the case explicit waits exist for: the table element may be present in the DOM immediately (an empty <table> shell) while its rows are still being populated by an async request, so waiting for the ELEMENT to exist is not sufficient; the wait condition needs to check the actual row COUNT. WebDriverWait.until accepts any callable that takes the driver and returns a truthy value once the condition holds, which is exactly what a len(rows) > 0 lambda expresses. Once rows are present, searching them for a specific value is ordinary text matching, not something that needs its own wait, since by that point the data is already loaded.

Worked example

from selenium.webdriver.common.by import By
from selenium.webdriver.support.ui import WebDriverWait

class OrdersTable:
    ROWS = (By.CSS_SELECTOR, "table#orders tbody tr")

    def __init__(self, driver):
        self.driver = driver
        self.wait = WebDriverWait(driver, 10)

    def wait_until_loaded(self):
        self.wait.until(lambda d: len(d.find_elements(*self.ROWS)) > 0)

    def has_row_with_value(self, value):
        rows = self.driver.find_elements(*self.ROWS)
        return any(value in row.text for row in rows)

@pytest.fixture
def driver():
    from unittest.mock import MagicMock
    d = MagicMock()
    row = MagicMock()
    row.text = "Order #12345 - Shipped"
    d.find_elements.return_value = [row]
    return d

def test_table_contains_order(driver):
    table = OrdersTable(driver)
    table.wait_until_loaded()
    assert table.has_row_with_value("Order #12345") is True

Verified against a mocked driver returning one row whose text is "Order #12345 - Shipped" (no browser is available here, so the driver fixture above stubs find_elements instead of launching a real one):

$ pytest test_ajax_table.py -v
test_table_contains_order PASSED
1 passed in 0.10s

The wait condition (len(rows) > 0) and the search ("Order #12345" in row.text) both executed correctly against the real WebDriverWait/By API, with only the underlying find_elements calls mocked.

Trade-offs and pitfalls

Waiting only for the table ELEMENT to exist (presence_of_element_located on the table itself) rather than for its ROWS to be populated is the single most common mistake on AJAX content: the test passes the wait immediately because the empty table shell is already in the DOM, then fails the search a moment later because the rows have not loaded yet, which reads as a flaky test rather than the wait-condition bug it actually is. A second pitfall is searching row.text for a substring match on data that could plausibly appear in an unrelated row (a customer name that happens to contain the order number's digits, for instance); scoping to a specific column via a more precise CSS selector avoids a false positive once the table has enough rows to make collisions realistic.

Direct answer

Reach for low-level techniques only when the standard WebDriver API and ordinary JavaScript injection genuinely cannot express the interaction, because low-level techniques trade a real cost (platform dependence, brittleness, harder debugging) for capability the standard API does not offer; for everything the standard API CAN do, it should be preferred, and low-level techniques should be the deliberate exception, not the default.

Structured elaboration

The standard WebDriver Actions API (mouse/keyboard action chains) and execute_script cover the large majority of real interactions: clicks, typing, hovers, scrolling, reading computed values. Native OS-level tooling (the W3C Actions API's lower-level primitives, a platform automation library, or something like Sikuli/Robot-class image or coordinate-based automation) becomes necessary specifically when the interaction is not expressible purely inside the browser's own DOM/JS model: genuine HTML5 drag-and-drop (which many browsers implement via native OS drag events that a synthetic DOM event does not fully replicate), complex multi-touch gestures on mobile, or a native OS file-picker/print dialog that lives outside the browser's own DOM entirely and therefore cannot be reached by any in-page JavaScript or WebDriver DOM command.

Each named example maps to a specific reason the standard layer falls short: HTML5 drag-and-drop often requires real, OS-level drag events firing in the right sequence, which is why Actions.dragAndDrop sometimes fails silently and a JavaScript-simulated event or a lower-level native event sequence is needed instead. Complex gestures (pinch-zoom, multi-finger swipe on a mobile emulator) are not expressible as a single WebDriver Actions call and need the platform's own gesture APIs. Native dialogs (a browser's built-in file picker, as opposed to a styled in-page upload widget) exist entirely outside the DOM the browser exposes to WebDriver/JavaScript, so no execute_script call can reach them at all; either OS-level tooling or, more commonly, a WebDriver-native workaround (sending the file path directly to the underlying <input type="file"> element instead of opening the OS picker) sidesteps the problem rather than fighting it head-on.

Worked example

A decision an interviewer is really listening for:

• Task: click a button. Standard API (element.click()). No escalation needed.
• Task: drag a card between two Kanban columns implemented with native HTML5 drag events. Actions.dragAndDrop first; if it silently does not move the element (a known gap for some HTML5 implementations), fall back to a JavaScript event-simulation approach before reaching for OS-level native event injection, which is the last resort, not the first.
• Task: upload a file via a custom-styled button that internally still uses <input type="file">. This LOOKS like it needs a native OS dialog automation tool, but the standard API workaround (send the file path directly to the underlying, often-hidden, <input type="file"> element) avoids the OS layer entirely and should be tried before any native-dialog tooling.
• Task: a print dialog or a true OS-level "Save As" file browser with no underlying <input> to target. No DOM-level workaround exists; this is the genuine case for native OS automation tooling (or, in many real suites, a decision to NOT automate that one step and instead verify the file was produced by checking the filesystem directly).

Trade-offs and pitfalls

The costliest mistake is reaching for native OS-level automation (Robot class, platform-specific tools) too early, before confirming there is no DOM-level or JavaScript-level workaround. Native automation is inherently less portable across CI environments (headless containers frequently cannot drive real OS-level input events at all), harder to debug (failures do not produce a helpful WebDriver-style error, since you are outside its protocol entirely), and slower to run. The correct escalation order is standard API, then JavaScript injection/simulation, then native OS-level tooling only when the previous two are demonstrably insufficient, and a senior candidate should be able to justify each step of that escalation rather than jumping straight to the most powerful (and most fragile) tool.