Automated a11y testing in CI and its limits

WHAT THIS TESTS This checks whether you can operationalize accessibility in CI and whether you understand that automation has a hard ceiling.

A GOOD ANSWER COVERS Wire automated checks into the pipeline at several layers. Use a linter such as eslint-plugin-jsx-a11y to catch static issues at commit time. Run axe-core against each component, typically through its Storybook stories or in jest or testing-library tests, so violations fail the build. Add color-contrast assertions and snapshot the rendered ARIA structure. Gate pull requests so new violations cannot merge, and report results visibly. Crucially, state the limits: automated tools reliably detect only a portion of WCAG issues, roughly a minority of real-world barriers. They cannot evaluate whether focus order is logical, whether labels are meaningful, whether content makes sense to a screen reader, or whether interactions feel usable. So complement automation with manual keyboard navigation, screen-reader testing, and periodic audits.

COMMON WRONG ANSWERS Treating a passing axe scan as proof of compliance. Running checks only locally and not gating CI. Ignoring that contrast and dynamic content need extra handling. Forgetting manual testing entirely.

LIKELY FOLLOW-UPS Which issues can automation never catch? How do you stop flaky a11y tests? How do you budget manual audit time across many components?

ONE CONCRETE EXAMPLE A pull request adds a Menu component. CI runs jest-axe on its stories and fails because triggers lack accessible names. The dev fixes the labels and the build passes. Before release, a human still tests arrow-key navigation and a screen reader, catching that focus does not return to the trigger on close, which axe never reported.