Why Five Engines Catch More Than One: How Lumi's Scanner Works

When we built Lumi, we started with a question: why do different accessibility scanners give different results for the same page?

The answer turned out to be straightforward. Each scanner checks different things using different methods. No single engine covers everything. So we built a scanner that runs five engines in parallel and deduplicates the results.

This post explains what each engine does, what it catches that others miss, and how the combination works.

The five engines

1. axe-core (Deque)

axe-core is the industry standard for automated accessibility testing. It is open source, well-maintained, and checks approximately 84 rules covering WCAG 2.0, 2.1, and 2.2 success criteria.

What it catches well:

• Missing or empty alt text
• Form inputs without labels
• Invalid ARIA attributes and roles
• Colour contrast failures (text and non-text)
• Duplicate IDs
• Missing document language
• Heading hierarchy issues

What it misses:

• Keyboard accessibility (it analyses the DOM, it does not tab through the page)
• Focus visibility quality (it can check if a focus style exists, not whether it is visible enough)
• Dynamic content that only appears after interaction
• Touch target sizes
• Content behind scrollable regions

axe-core is excellent at what it does. The limitation is the method: static DOM analysis cannot test interactive behaviour.

2. HTML_CodeSniffer

HTML_CodeSniffer (HTMLCS) is an older but complementary engine maintained by Squiz. It checks against WCAG 2.1 guidelines with a different rule set than axe-core.

What it catches that axe-core sometimes misses:

• Advisory and warning-level issues that axe-core classifies as "incomplete" or skips entirely
• Specific HTML validity issues that affect assistive technology
• Content language changes within a page
• Certain table accessibility patterns

The overlap between axe-core and HTMLCS is roughly 60%. The remaining 40% is unique to each engine. Running both and deduplicating the results catches more than either alone.

3. Keyboard testing engine

This is Lumi's custom engine built on Playwright. Unlike axe-core and HTMLCS, which analyse the DOM statically, the keyboard engine actually navigates the page.

What it tests:

• Tab order - does the tab sequence follow a logical reading order? Are there elements that receive focus but should not, or interactive elements that are skipped?
• Keyboard traps - can the user tab into and out of every component? Modals, dropdown menus, embedded videos, and custom widgets are common trap points.
• Skip navigation - does the page provide a mechanism to bypass repeated navigation blocks?
• Focus visibility - when an element receives focus, is the indicator visible? The engine screenshots focused elements and analyses the visual difference.
• Focus not obscured - is the focused element hidden behind a sticky header, cookie banner, or fixed footer? This is a WCAG 2.2 Level AA criterion (2.4.11) that static DOM analysis cannot check.

No other scanner on the market tests these by actually tabbing through the page. They are either skipped entirely or checked through DOM heuristics that miss real-world failures.

4. Lumi custom rules

These are 66 proprietary rules we wrote to cover gaps in the other engines. Each rule maps to a specific WCAG success criterion.

Examples of what custom rules catch:

• Autoplaying media without a pause mechanism (WCAG 1.4.2)
• Target sizes below the 24x24 CSS pixel minimum (WCAG 2.5.8)
• Missing visible labels where only placeholder text is used (WCAG 3.3.2)
• Interactive elements with insufficient non-text contrast on borders and backgrounds (WCAG 1.4.11)
• Empty links and buttons (elements that are focusable but announce nothing to screen readers)
• Images of text where real text could be used (WCAG 1.4.5)
• Redundant ARIA roles that duplicate native HTML semantics
• Form fields that change context on input without warning (WCAG 3.2.2)

The full list of Lumi's 66 custom rules with their WCAG mappings is published on our checks page.

5. AI visual analysis

The newest engine uses Claude to analyse screenshots of the rendered page. This catches issues that require visual judgement rather than DOM inspection.

What it checks:

• Alt text quality (not just presence but whether the alt text is actually descriptive)
• Visual content that appears as images of text
• Decorative images incorrectly marked as meaningful
• Complex graphics without adequate text alternatives

AI analysis is the most experimental of the five engines. It runs on every scan but its findings are flagged with lower confidence than deterministic rule checks. We are transparent about this because we think the accuracy-honesty trade-off matters more than inflated detection numbers.

How deduplication works

Running five engines on the same page produces duplicates. The same missing alt text will be flagged by axe-core, HTMLCS, and potentially a custom rule.

Lumi deduplicates by normalising each finding to a canonical form: the WCAG criterion, the page URL, and the DOM element (identified by CSS selector). When two engines report the same issue on the same element, the results are merged into a single finding attributed to the engine with the highest confidence.

This means the issue count you see in Lumi is the deduplicated total, not the raw output of five engines summed together.

What the numbers show

Across our benchmark testing on 100 sites:

• axe-core alone found an average of 47 issues per site
• axe-core plus HTMLCS found an average of 58 issues per site (23% more)
• All five engines found an average of 72 issues per site (53% more than axe-core alone)

The additional issues are not noise. They are real accessibility barriers - keyboard traps, obscured focus indicators, insufficient target sizes, and autoplaying media - that single-engine scanners structurally cannot detect.

What automation still misses

Five engines are better than one. But automated scanning, regardless of how many engines you run, catches approximately 30-40% of all WCAG issues.

The remaining 60-70% requires human judgement:

• Is the alt text meaningful, not just present?
• Does the page make sense to a screen reader user in context?
• Are error messages helpful?
• Is the reading order logical for someone who cannot see the layout?
• Does the content use language appropriate for its audience?

This is why we recommend pairing automated monitoring with periodic manual audits. Lumi catches what automation can catch, more thoroughly than single-engine tools. For everything else, the Livana audit team tests with real assistive technology.

Scan any site free at lumi.livana.io to see the five-engine difference.