How much of the accessibility criteria does an automated check actually cover?

About a third, and by most estimates between 30 and 40 percent. An automated tool is excellent at finding anything that can be phrased as an unambiguous rule: an image with no alternative text, contrast under the threshold, a form field with no label, a skipped heading level. It cannot judge whether the alternative text is right, whether the keyboard order makes sense, or whether a message that appeared on screen was actually announced.

So a Lighthouse 100 means nothing?

It means something, it just does not mean "meets the standard". It means everything checkable by machine is in order, which is necessary and not sufficient. A site can score 100 and still be impossible to navigate by keyboard, or carry alternative text that reads "image". Neither of those is tested by the tool.

If I only have five minutes, which manual check matters most?

Unplug the mouse and walk the page with Tab alone, start to finish. Three questions: can you reach every button and link, can you see where the focus is at all times, and is the order sensible rather than jumping around. If the focus marker disappears somewhere, someone turned it off in the design, and that is a level AA failure.

When is it appropriate to sign an accessibility statement?

After the manual pass has been done too, not only the scan. The statement is a public document declaring what was tested and what the known limitations are. A statement signed on the strength of a scan alone declares something that was never really checked, which is the opposite of its purpose.

What should I ask a supplier who tells me the site is accessible?

Three questions. Which tools were used and at which screen widths. Whether keyboard behaviour was tested or only a scan was run. And whether there is a test document you can see, with what was found, what was fixed and what is still open. A supplier who has that document will show it immediately, and that is the most telling answer of the three.

A 100 Accessibility Score Is Not Compliance

There is a very pleasant moment in building a website: you run an automated accessibility check and get a 100. Green, clean, zero errors. The temptation to tick the box and move on is enormous.

The problem is that this number answers a far narrower question than it appears to.

What an automated tool actually checks

A tool like Lighthouse or axe is excellent at one thing: finding anything that can be phrased as an unambiguous rule a computer can decide. An image with no alternative text. Contrast of 3.9 where 4.5 is required. A form field with no associated label. A jump from an H2 straight to an H4.

These are real, useful checks and they save hours. But notice what they have in common: they ask whether something exists, not whether it is right.

The accepted industry estimate is that automated testing covers between 30 and 40 percent of the standard’s criteria. The rest needs a person, not because the tools are poor but because some questions simply cannot be automated.

What it cannot catch, and why

Whether the alternative text is correct. The tool checks that an alt is present. It cannot tell whether it says “the team greeting a client at a first meeting” or simply “image”. Both pass, and only one helps anybody.

Whether the keyboard order makes sense. You can check that an element is reachable by Tab, but not that the order matches what the eye sees. In Hebrew this is a particular trap: certain layout rules change the visual order without changing the order in the code, and focus starts hopping between areas of the page.

Whether a message was announced. When a visitor submits a form and “Thanks, we’ll be in touch” appears, someone using a screen reader needs to hear it. If the sentence simply appeared on screen without the right marking, it is swallowed silently. The tool sees valid text and has no idea it arrived just now.

Whether something that looks right actually works. This is the sneakiest category, and we ran into it ourselves.

What happened when we tested ourselves

We scanned this site against WCAG 2.2 at level AA, at two screen widths, across every page. The result was zero findings. At that point we could have stopped, written “tested and found compliant”, and moved on.

Then we went through it by hand, and this is what turned up:

The skip link did nothing. That link exists so someone navigating by keyboard can jump past the menu straight to the content. Ours appeared in the right place, looked perfect, and on activation focus simply stayed where it was. The next Tab returned the visitor to the navigation, which is precisely what the link exists to skip. No tool can catch this, because as far as the code is concerned everything is in order.

A request for reduced motion was only half honoured. Anyone who had told their operating system they want fewer animations got our entrance animations disabled, while three drifting background shapes kept moving without pause. Those are the ones that matter, because they never end.

The bot answered and a screen reader said nothing. Replies from our AI assistant appeared on screen without being announced. Same for the result screen of our sizing wizard.

Four defects, after a scan that returned zero.

What this means for an accessibility statement

An accessibility statement is a public document saying what was tested and what the known limitations are. A statement resting on a scan alone declares something that was never really tested.

So our rule is simple: the scan is the entry condition, not the finish line. After it comes a list of roughly 20 to 30 minutes of manual work, and only then does anyone sign. At the end of every audit we write a document recording who tested, when, with which tools, what was found, what was fixed, and what has not been tested. Ours says in plain words that we did not use a real screen reader.

The five-minute check you can run now

If you only do one thing from the manual list, make it this one. Move the mouse aside and walk the page using the Tab key alone, start to finish. Three questions:

Can you reach everything? Every link, every button, every field. If a control can only be reached with a mouse, it does not exist for some people.

Can you always see where you are? There has to be a clear marker on the active element. If it disappears partway, someone switched it off in the design because “it looks ugly”. That is a failure at the binding level, and it also makes the site impossible to navigate.

Is the order sensible? Focus should flow the way the eye reads. If it jumps from the heading to the footer and back, there is a gap between the visual order and the order in the code.

And if you have two minutes more: open a menu or a dialog, confirm you can leave it with the Esc key, and that focus is not trapped inside. A focus trap is one of the most severe defects there is, because it does not slow a visitor down, it stops them completely.

In short

A score of 100 is a good sign, and it is not a certificate. It says everything a machine can check is in order, which is a third of the story.

And if you are weighing up a supplier, this question is worth more than all the rest: show me the test document. Anyone who has one will show it immediately.

To be clear: everything here is general information, not legal advice.

Want to know what was actually tested on your site? Let’s talk ↗