Accessibility Testing Questions

Situation: I need to ensure form error messages and interactive error controls (e.g., "retry", "dismiss") are usable by keyboard and screen-reader users, and prevent regressions via CI.

Plan (high level):

• Automated unit/accessibility checks for component-level behavior.
• End-to-end tests that exercise keyboard flows and screen-reader-relevant attributes.
• Run these tests in CI and fail the build on regressions.

Tools:

• axe-core / jest-axe for unit/component tests
• Playwright (or Cypress) + @axe-core/playwright for E2E + keyboard automation
• Testing Library (@testing-library/react) for component assertions
• CI: GitHub Actions / GitLab CI to run headless browser tests
• Optional manual verification: NVDA/VoiceOver for spot checks

Example checks / assertions:

• Error message exists in DOM and is programmatically associated:
  • expect(input).toHaveAttribute('aria-describedby', 'email-error')
  • expect(document.getElementById('email-error').textContent).toMatch(/required/)
• Error is announced to screen readers:
  • error container has role="alert" or aria-live="polite"
  • expect(errorEl).toHaveAttribute('role','alert')
• Keyboard focus and interactive control behavior:
  • Tab to input, submit invalid -> focus moves to first invalid field or to the error summary
  • Simulate keyboard: page.keyboard.press('Tab'); page.keyboard.press('Enter'); expect(page.evaluate(() => document.activeElement.id)).toBe('email')
• Interactive controls are reachable by keyboard and accessible:
  • expect(button).toHaveAttribute('tabindex','0') or naturally tabbable; expect(button).toHaveAccessibleName('Retry sending email')
• Axe assertions:
  • await expect(await axe.run(page)).toHaveNoViolations() (or assert zero violations for relevant rules like aria-allowed-attr, color-contrast, etc.)

Example Playwright + axe snippet:

// playwright-axe example
const { test, expect } = require('@playwright/test');
const { injectAxe, checkA11y } = require('axe-playwright');

test('error messages are accessible', async ({ page }) => {
  await page.goto('http://localhost:3000/form');
  await page.click('button[type=submit]'); // trigger validation
  await injectAxe(page);
  await checkA11y(page, null, {
    detailedReport: true,
    rules: { 'color-contrast': { enabled: true } }
  });
  // keyboard focus assertion
  await page.keyboard.press('Tab');
  expect(await page.evaluate(() => document.activeElement.getAttribute('aria-describedby'))).not.toBeNull();
  // check role/aria-live
  const role = await page.getAttribute('#email-error','role');
  expect(role).toBe('alert');
});

CI integration:

• Add a job that installs dependencies, starts the app (or uses test build), runs Playwright tests headless.
• Fail on any test or axe violation. Use caching and parallel jobs to keep runtime reasonable.
• Consider baselining: if temporary known issues exist, record them and require explicit triage before allowing CI to ignore.
• Run full accessibility E2E nightly if heavy; run lightweight unit/axe checks on every PR.

Why this works:

• Combines programmatic checks (aria relationships, role, focus) with automated screen-reader–relevant testing (aria-live/role and keyboard flows) and axe rules to catch common a11y regressions, and enforces them in CI so no manual QA step is required for regressions.

Approach summary: target WCAG 2.1 AA, ensure semantic markup + keyboard operability + ARIA where needed, test automatically and manually, gate via CI and PR checklist.

Automated tests

• Unit/axe: use jest-axe to run axe against component render.
• E2E/axe: run cypress-axe in integration tests to test real DOM and interactions (focus, tooltips).
• CI/CLI: run axe-core CLI or pa11y-ci and Lighthouse CI to fail builds on violations.

Example (jest-axe):

import React from 'react';
import { render } from '@testing-library/react';
import { toHaveNoViolations } from 'jest-axe';
import { axe } from 'jest-axe';
import Chart from './Chart';
expect.extend(toHaveNoViolations);

test('chart is accessible', async () => {
  const { container } = render(<Chart data={sample} />);
  const results = await axe(container);
  expect(results).toHaveNoViolations();
});

Manual keyboard & screen reader checks

• Keyboard: Tab/Shift+Tab, Arrow keys, Enter/Space to activate points/legend; ensure logical focus order, visible focus styles, skip-to controls for dense charts.
• Screen reader: NVDA/JAWS (Windows), VoiceOver (mac/iOS). Verify announcements for chart role, accessible name/description, data point labels on focus, summary for large datasets.
• Mobile: TalkBack/VoiceOver checks for touch focus/gesture semantics.

Tools

• axe DevTools / browser extension, Accessibility Insights, Lighthouse, Color Contrast Analyzer, NVDA/VoiceOver, cypress-axe, jest-axe, pa11y.

Acceptance criteria (examples)

• No automated axe violations at AA level for component interactions.
• Keyboard: all interactive elements reachable and operable; focus visible.
• Screen reader: chart has accessible name/description; selecting a data point announces value, label, and index.
• Color contrast >= 4.5:1 for text; non-text contrast >=3:1 where applicable.
• Performance: tooltips/readouts do not block navigation.

CI and PR integration

• Add jest-axe unit tests and cypress-axe E2E to CI; fail build on regressions.
• Run Lighthouse/pa11y in CI weekly or per-merge for full pages.
• Enforce accessibility linting (eslint-plugin-jsx-a11y) and pre-commit hooks.
• PR template includes an a11y checklist: automated tests passing, keyboard + screen reader verification notes, screenshots/GIFs of keyboard flow or VoiceOver output for complex changes.
• Use incremental remediation: triage axe violations in PR, require dev to fix or document exception with product/UX sign-off.

Reasoning: Combining automated and manual checks catches structural issues (axe), runtime/interaction issues (E2E + manual keyboard), and real assistive-technology behaviors (screen reader). CI + PR policies ensure regressions are caught early and accessibility is treated as part of quality, not an afterthought.