Are 98 percent of the websites not accessible - a critic on the WebAim Million study

98 percent of most used websites not accessible - the news made the rounds on Twitter and relevant accessibility channels a few months ago. Great stuff, especially if you only read headlines. Personally, I don't find the WebAIM study meaningful for numerous reasons. That's what I want to lay out in this post.
Briefly to explain, in professional circles we don't talk about accessibility, we talk about conformance. Conformance means that a certain standard has been met, for example WCAG 2.1 at level AA. Since the term "accessible" is not firmly defined for websites, this makeshift is always necessary.
And here is already the first mistake: WebAIM has performed its audit at compliance level AA. AA, however, is the standard for public organizations. Private organizations usually aim for A, not AA. At the A level, for example, there are no specifications for minimum contrasts. However, if you are not required to meet AA and only strive for A, the AA rating is not meaningful. It is as if I were rating amateur athletes by the standards of professional athletes.

Methodology

. The one million websites were reviewed automatically using WebAIM's WAVE tool. There is not much more to say about the methodology.

Automated tools are limited to not helpful

. What I have currently seen of automated accessibility testing has not convinced me. Now WebAIM Wave may still be one of the more sophisticated tools, but unfortunately that's not saying anything either. There are things you can measure automatically like the presence of certain HTML elements, ARIA attributes, labels, alternate text and some contrasts. But the list of things they can't evaluate is longer. These include whether alternate texts make sense, whether ARIA is used wisely, whether texts or form elements are labeled correctly.
In short: Whether Wave displays errors or not is completely irrelevant. A lazy but smart developer will run the tool over, iron out the errors and get his site compliant without having improved accessibility one bit.
As WebAIM itself notes, websites are becoming more and more complex. However, I assume that many websites, especially from the Anglo-American world, have accessibility on their radar. That is, they care about alternative texts or meaningful link descriptions. However, it is sometimes not possible to take these factors into account for externally embedded content.
A large proportion of errors are likely to be due to such embedded content: This is, for example, social media content or advertising. If you go by WebAIM, you should probably leave out such content, since you can't make it accessible. This is more likely to scare people away from accessibility.
Let's take a closer look at the errors:

• 86 percent with errors in contrast: As noted above, not an AA criterion.
• 66 percent images with missing alternative texts: This probably involves externally embedded content over which one has no great influence; the same applies to links without text.
• 53 percent with missing form labels: Indeed annoying, but this can only be judged in context. If it is about the search field and there is only one field, this error is not so bad.
• 28 percent missing document language - completely irrelevant, since most users of the web pages are likely to be native speakers.

No page is without error

. The one million most visited websites are probably each maintained by larger teams. There it can happen again and again that individual editors make mistakes: Be it the incorrect integration of a widget, the wrong nesting of headings or the forgetting of the alternative text. Let him who is without fault cast the first stone at WebAIM. That is, even a single mistake by an editor can cause the website to fail WCAG. You may find this sensible, but it is not practically relevant.
So 98 percent of all websites have errors, it should be closer to 100 percent. Anyone who has ever evaluated websites knows that you can find errors if you specifically look for them.
In the end, however, it is not about technical perfection, but about people with disabilities being able to use the website. The WebAIM study actually says nothing about that.

Motivate or demotivate

. A client tried to persuade me to mention the study in one of the training sessions. I refused, for the reasons mentioned above. But also because I think the signal is fatal. The study can show that others are not doing better than you and then motivate you to do more.
In my opinion, however, it has a demoti