Accessibility Testing Questions

Approach summary
Automate accessibility checks for dynamic UI by combining programmatic axe-core scans with deterministic focus/navigation assertions. Use automated scans to catch semantic/attribute issues and targeted functional tests to validate ARIA live regions and focus management after re-renders.

Automated + manual strategy

• Use axe-core (integrated into Playwright/Puppeteer or Selenium) to run post-action scans whenever the UI mutates (after network responses, state updates, modal opens, etc.).
• Add focused functional tests that simulate keyboard users: Tab/Shift+Tab, Enter/Escape, arrow keys, and ensure focus lands where expected.
• Validate ARIA live regions by injecting a mutation observer or polling for live region text updates and asserting announced content and polite/assertive roles.

Sample Playwright + axe-core snippet

// javascript
const { injectAxe, checkA11y } = require('axe-playwright');
await page.goto(url);
await injectAxe(page);
// perform action that causes dynamic content/focus change
await page.click('#open-dialog');
// run axe after UI stabilizes
await checkA11y(page, null, { detailedReport: true });

Focus/navigation example

• After creating an item, assert focus moves to the new item's primary control:

const focused = await page.evaluate(() => document.activeElement.id);
expect(focused).toBe('item-123-edit-btn');

When to fail vs warn

• Fail: violations that block keyboard/screen-reader interaction (missing focusable controls, focus traps, incorrect ARIA roles, critical contrast, missing live region updates when content is only visually conveyed).
• Warn: low-severity issues or best-practice improvements (suggested ARIA tweaks, non-critical contrast, redundant roles) that should be reviewed but don’t prevent access.
• Implement CI policy: fail pipeline on any axe “serious” or “critical” violations and on deterministic focus assertion failures; log “moderate/minor” as warnings with links to DOM snapshots and repro steps.

Why this works
Combining semantic scans with deterministic keyboard/focus assertions captures both static attribute problems and runtime interaction regressions that automated scanners alone can miss. Continuous CI enforcement ensures regressions are caught early while triaging warns keeps noise manageable.

Approach overview
I would treat automated a11y checks as first-class tests in the UI pipeline: fast checks on every PR for rapid feedback, deeper scans nightly, and full-site audits on a cadence (weekly or before releases).

Where to run checks

• PRs: run lightweight axe-core or Lighthouse CI on affected pages/components; surface failures as GitHub/GitLab annotations so devs fix during code review.
• Nightly: run broader site-wide checks, multiple viewports, and generate historical reports.
• Pre-release: run full Lighthouse audits and export artifacts.

Handling noisy results & false positives

• Classify rules by severity and confidence; fail PRs only on high-severity deterministic failures.
• Use baseline/approval mechanism: allow known issues to be recorded with IDs and expiration; store suppression lists in repo.
• Improve selectors/tests to avoid transient DOM states; retry or wait for stable state before scanning.
• Add comment with failing HTML snippets and links to failing element for quick triage.

Combining automated + manual

• Automated for regression, coverage, and blocking issues.
• Manual review for keyboard flows, screen reader experience, cognitive and content clarity.
• Establish an a11y triage workflow: automation flags → accessibility champion reviews → assign dev/UX fixes → update baseline if false positive.
• Track metrics (open a11y defects, time-to-fix) and run periodic paired reviews (tester + developer) for complex flows.

Example tools: axe-core in Selenium/Cypress, axe-cli for node, Lighthouse CI in GitHub Actions, and reports visualized in Allure/HTML for reviewers.