Automation Testing and Debugging Questions

Focuses on methods and tooling for testing and debugging automated scripts and applications across environments and layers. Includes diagnosing flaky tests, analyzing test failures, reading and interpreting logs, setting breakpoints, using browser developer tools, capturing screenshots and video recordings, and using remote debugging approaches. Covers systematic root cause analysis to determine whether failures stem from test code, application code, environment or infrastructure, and strategies for isolating problems such as component level testing and reproducible minimal examples. Addresses cross layer troubleshooting across frontend, application programming interface, database and network components as well as platform specific testing considerations such as emulator versus real device behavior and mobile device operating system differences. Also includes best practices for test design, logging and monitoring, making test failures actionable for developers, and troubleshooting automation within continuous integration and continuous delivery pipelines and shared environments.

EasyTechnical

50 practiced

Explain how to set breakpoints and debug Node.js automated tests (Mocha or Jest) both locally and inside a Docker container using VS Code. Include sample launch.json entries for local and remote attach scenarios, and the steps to attach to a running container process.

Sample Answer

**Brief approach**Explain how to start tests with the Node inspector, create VS Code launch configurations for local debug and remote attach, and the steps to attach to a process running inside Docker.

**Start tests with inspector**- Mocha: node --inspect-brk=9229 ./node_modules/.bin/mocha --timeout 999999 --recursive test- Jest: node --inspect-brk=9229 ./node_modules/.bin/jest --runInBand

Use --inspect for non-breaking start; --inspect-brk pauses on first line so you can set breakpoints.

**VS Code launch.json — Local run (launch program)**

json

{
  "version": "0.2.0",
  "configurations": [
    {
      "type": "pwa-node",
      "request": "launch",
      "name": "Debug Mocha (launch)",
      "program": "${workspaceFolder}/node_modules/mocha/bin/_mocha",
      "args": ["--timeout", "999999", "--recursive", "test"],
      "cwd": "${workspaceFolder}",
      "console": "integratedTerminal",
      "internalConsoleOptions": "neverOpen",
      "skipFiles": ["<node_internals>/**"]
    }
  ]
}

**VS Code launch.json — Attach to running inspector (local or container port forwarded)**

json

{
  "type": "pwa-node",
  "request": "attach",
  "name": "Attach to Node",
  "address": "localhost",
  "port": 9229,
  "localRoot": "${workspaceFolder}",
  "remoteRoot": "/usr/src/app",
  "skipFiles": ["<node_internals>/**"]
}

**Attach to running Docker container**1. Start container exposing 9229: docker run -it -p 9229:9229 -v $(pwd):/usr/src/app my-image - Or if already running, restart test process with inspector: docker exec -it <container> sh -c "node --inspect=0.0.0.0:9229 ./node_modules/.bin/jest --runInBand"2. Verify inspector: docker ps and curl localhost:9229/json/list or nc -vz localhost 92293. In VS Code pick the "Attach to Node" config and start it. Ensure localRoot maps to source and remoteRoot matches container path.4. Set breakpoints in VS Code; use --inspect-brk to stop on first line if breakpoints are missed.

**Tips & pitfalls**- Use --runInBand (Jest) or single process Mocha to avoid child process complexity.- For child processes, use --inspect and attach to child PID or configure NODE_OPTIONS="--inspect=9229".- Map source paths correctly (localRoot/remoteRoot) to avoid “source not found” issues.- In CI, prefer headless logs; debugger is for dev troubleshooting.

EasyTechnical

27 practiced

Compare implicit waits, explicit waits, and fluent waits in Selenium WebDriver. For each: explain behavior, typical use cases, pitfalls (e.g., implicit+explicit interaction causing unexpected delays), and recommend best practices or patterns to ensure stable UI tests.

HardTechnical

48 practiced

You maintain a large test suite with slow E2E tests. Propose a plan to reduce total runtime while preserving meaningful coverage: discuss parallelization, shard strategies, change-based test selection, splitting smoke/regression/nightly suites, caching, virtualization vs real integration, and metrics to validate that coverage and defect detection remain acceptable.

Sample Answer

**High-level goal**Reduce E2E wall-clock time while preserving meaningful coverage and defect detection rate so developers get fast, reliable feedback.

**1) Clarify constraints**- Max concurrent runners, test flakiness budget, acceptable feedback windows (PR vs nightly), required integrations (payments, 3rd-party).

**2) Parallelization & sharding**- Run tests in parallel across multiple agents/containers.- Shard by independent test suites (feature area) and by test runtime: use historical durations to create equal-time shards (balanced buckets).- Use deterministic shard assignment (hash test id) to keep cache locality and reproducibility.

**3) Change-based test selection**- On PRs, run a fast subset: impacted-area unit + integration + targeted E2E smoke tests mapped by ownership matrix and feature-to-test tags.- If diff touches infra/critical flows, escalate to full E2E.- Keep an allowlist of high-risk tests that always run.

**4) Suite split: smoke/regression/nightly**- PR pipeline: quick smoke (~5–10 min) + fast sharded integration.- Merge/regression: broader regression suite run in parallel (~30–60 min).- Nightly: full, exhaustive E2E including long, flaky, or costly tests.

**5) Caching & test setup optimization**- Cache build artifacts, docker layers, DB snapshots.- Use lightweight fixtures and reset state via snapshots rather than full migrations.- Reuse environment instances where safe.

**6) Virtualization vs real integration**- Mock low-risk 3rd-party calls in PRs; run end-to-end against staging with real integrations in nightly/regression.- For hardware or critical flows, keep a small real-integration pane on PRs (sampled).

**7) Flakiness & reliability**- Quarantine flaky tests with a triage workflow (mark, fix, or move to nightly).- Add retries only with logging and fail-fast thresholds to avoid masking issues.

**8) Metrics to validate**- Test suite wall-clock and median PR feedback time.- Per-test historical pass rate, mean time to detect defect, escaped defects (prod regressions) attributable to test gaps.- Coverage mapping: feature-to-test ratio and percentage of changed code covered by E2E over time.- Cost per run (agent-minutes) and ROI (defects prevented).- Gate improvements by maintaining or improving defect-detection rate while reducing PR feedback time.

**9) Rollout & governance**- Start with pilot: balance shards for a few teams, validate metrics for 2–4 weeks, iterate.- Document tagging and ownership, automated mapping for change-based selection.- Regular reviews: flaky-test backlog, coverage gaps, and cost trade-offs.

Result: faster, reliable feedback loop for devs, preserved critical coverage via sampling and nightly full runs, and measurable KPIs to ensure quality isn’t degraded.

MediumTechnical

24 practiced

A mobile automation test passes on Android 12 devices but fails on Android 11 with a UI element not found. Describe a step-by-step investigation plan to determine whether the problem is due to app code, OS differences (WebView/Chrome versions), layout rendering differences, or test framework behavior, and how you'd prove the root cause.

Sample Answer

**Goal**: Systematically determine whether the “element not found on Android 11” is app code, OS/WebView/Chrome differences, layout rendering, or test framework timing/locator issues — and prove the root cause.

1. **Reproduce & collect evidence**- Run failing test on an Android 11 device and passing test on Android 12; capture full Appium/UIAutomator/WebDriver logs, adb logcat, screenshots, screen recordings, and page source (getPageSource) at failure time.- Record exact device models, Android patch levels, Chrome/WebView versions, app APK version, and automation server/chromedriver versions.

2. **Compare DOM / view hierarchy**- Compare getPageSource / UI dump between Android11 vs Android12 at the same flow step. If the element exists but has different attributes/ids/xpaths, suspect layout/rendering or app code conditional logic.- Use Android Studio Layout Inspector or uiautomatorviewer to inspect runtime view on both OSes.

3. **Check timing / visibility**- Add explicit waits and increase timeouts. Instrument test to wait for ancestor visibility and log element search attempts. If adding waits fixes it, test framework timing is likely.

4. **Isolate WebView/Chrome differences (if hybrid/web)**- On Android11 device, open chrome://inspect and check WebView version and remote debug. Compare chromedriver compatibility with that WebView; run a simple Selenium script to find the same element using chromedriver matching that WebView. If chromedriver mismatch prevents element access, it’s OS/WebView related.

5. **Verify app behavior vs OS**- Install the same APK build on Android12 and Android11 and perform the flow manually. If manual Android11 also misses the element, app code or resource qualifiers (API-specific layouts) are suspect.- Build an instrumented debug APK with extra logging around view creation/visibility and deploy to Android11 to capture app-side reasons.

6. **Test framework behavior**- Run the same tests with a different automation backend (Espresso for native, or a different Appium server/chromedriver combo). If the problem disappears, framework compatibility is likely.

7. **Prove root cause**- For app code: show manual reproduction + debug logs showing view not created or different id on Android11.- For OS/WebView: show differing WebView/Chrome versions, chromedriver mismatch, and reproduce failure using standalone Selenium with same chromedriver; then show success after updating chromedriver or WebView.- For rendering/layout: show layout inspector screenshots showing element rendered off-screen or with different attributes on Android11.- For framework/timing: show that adding robust waits/retries or switching locator strategy fixes automated run while manual UI is fine.

8. **Remediation plan**- If app: file bug with stack traces and resource qualifiers; add compatibility fix.- If WebView/chromedriver: pin/update chromedriver in CI or vendor WebView.- If test fragility: change locators to stable attributes, add synchronization points, or switch framework.

Deliverables: comparison artifacts (page sources, logs, screenshots), repro steps, and a one-line root-cause statement with recommended fix and verification steps.

EasyTechnical

31 practiced

List the DevTools features and step-by-step actions you would use to debug a flaky browser E2E test that intermittently throws an "element not clickable" error. Cover DOM inspection, event listeners, layout, network, performance profiling, console errors, and how to capture reproducible traces for CI artifacts.

Sample Answer

**Brief approach (goal):** reproduce the failure reliably, identify root cause (timing, overlay, layout/paint, listener), then fix test or app. I’ll use Chrome DevTools + CI trace capture (CDP/Puppeteer/Selenium) to gather artifacts.

**Step‑by‑step actions**

1. Reproduce locally with same browser/version and headless/headful mode.2. DOM inspection (Elements) - Inspect target element, check computed styles, visibility, z‑index, pointer‑events. - Use “Force element state” (:hover/:active) and “Show DOM node properties” to confirm identity.3. Event listeners - In Elements → Event Listeners, list click handlers on element and ancestors; set breakpoints on mouse events. - Use Sources → Event Listener Breakpoints (Mouse → click) to pause and inspect call stack.4. Layout & rendering - Layers/Rendering panels: enable “Paint flashing”, FPS meter, and “Show composited layer borders” to detect repaints or overlays. - Check for animations/transitions; disable in Rendering → “Emulate CSS media prefers-reduced-motion” or set “Disable animations” to test flakiness.5. Network & timing - Network tab: confirm no late resources blocking layout (fonts, images). Replay failing run and look for 304/slow requests.6. Performance profiling - Record a Performance trace around the failing action; inspect main thread for long tasks, layout shifts (CLS), and input delay. - Look at Call Tree and Timing to see if click arrives during reflow or while overlay is being removed.7. Console & errors - Check Console for JS errors or warnings that could interrupt handlers. - Add console.assert/logs in app and test to surface timing.8. Reproducible CI artifacts - Capture HAR, screenshots (before/after), DOM snapshot, and full DevTools trace (chrome://tracing format) via CDP/Puppeteer or Selenium + Chrome DevTools Protocol. - Example: use Puppeteer’s page.tracing.start/stop to save trace; save network HAR via chrome-har; save screenshot on failure. - Attach artifacts to CI job for postmortem.9. Test fixes - Prefer app fixes; if test must change, replace brittle click with: waitForElementVisible, ensure element.isClickable by checking boundingClientRect and computed style, use element.click via JS only as last resort. - Add retries/timeouts conservatively; remove forced sleeps.

**Why these steps:** covers visibility, handlers, timing, rendering and network — the usual causes of intermittent “element not clickable.” Capturing traces and artifacts makes CI failures diagnosable without re-running locally.

Unlock Full Question Bank

Get access to hundreds of Automation Testing and Debugging interview questions and detailed answers.

Join thousands of developers preparing for their dream job.