Automated tests have some uses; but legal compliance demands
more: it requires that your site can be accessed and used by
disabled internet users – and this cannot be proved by an automated
tool.
In the following article, by Trenton Moss of usability and
accessibility specialists Webcredible, explains the risks
and limitations of automated tools.
The problems
Automated accessibility tools are useful because they can save
you a huge amount of time. Don't want to check images for alt text
on each and every page on your website? Run the site through an
automated tester and it'll do it all for you!
Automated accessibility testing tools have been around for a
long time and have historically been a useful way of checking
websites for accessibility. Bobby, one of the first and most
well-known automated accessibility testing tools, is now almost 10
years' old, and although it's no longer freely available, plenty of
other free tools such as WebXact and Wave do exist.
But are these tools a little too good to be true? Can you test a
website for accessibility so easily? No. There are a number of
underlying problems associated with using just automated tools to
test for accessibility:
Literal interpretation of guidelines
Any automated accessibility testing tool, being a piece of
software, doesn't have very much in the way of common sense. It
will interpret each and every accessibility guideline literally,
without bearing any other thought to what else is on the page.
The definition of the word guideline, according to
Dictionary.com, is “a rule or principle that provides guidance to
appropriate behaviour”. A guideline simply offers guidance to what
the best practice is it shouldn't just be applied without
regard to other factors.
For example, one of the W3C accessibility guidelines states that
a table summary should be provided for all tables. (This summary
doesn't appear on the screen, but it's read aloud to screen reader
users before reading through the table content.) Table summaries
are useful as they tell screen reader users what to expect in the
table. However, there may be a heading directly before the table
and it describes what the table is about. In this instance, this
summary is essentially useless as it will just repeat what the
previous heading said.
Can't check any content issues
The way that content is structured both on the page and across
the website is a massive part of accessibility. A website may be
perfectly coded and conform to the highest coding standards.
However, if its content is poorly structured, the site will prove
difficult to impossible for some special needs web users.
There are a number of important accessible content
considerations, none of which automated accessibility testing tools
can check for. Some of these important considerations include:
- Front-loading content so that each paragraph begins with the
conclusion
- Ensuring content has been broken down into manageable chunks
with descriptive sub-headings
- Using lists wherever appropriate
- Ensuring that plain and simple language is used
Can't check many coding issues
The vast number of accessibility guidelines tend to be related
to how the site is coded. Automated accessibility testing tools are
unfortunately unable to test for many of these too. Examples of
HTML-related accessibility considerations which these tools can't
check for include:
- Ensuring that text is real text and isn't embedded within
images
- Making sure that the site functions without the use of
JavaScript or Flash
- Providing equivalent text links if using server-side image
maps
- Ensuring that the structure within the HTML reflects the visual
appearance (e.g. headings are labelled as headings within the HTML
code)
Outdated guidelines are used
Automated accessibility testing tools generally use the W3C
accessibility guidelines, which by now are over five years old. As
such, a number of these guidelines are outdated and don't apply
anymore. In fact, some of them are now thought to hinder
accessibility rather than help, so it's best to totally ignore
those guidelines that have become out of date.
For example, an automated accessibility testing tool will
probably insist that form items contain default place holding text.
This requirement is only relevant to very old screen readers. It
may also insist that links need to be separated by non-link text.
Neither of these guidelines are relevant anymore and their
implementation could make accessibility worse rather than
better.
Most guidelines aren't properly checked
Automated accessibility tools can check for a number of
guidelines, and can tell you when a guideline isn't being adhered
to. However, when the tool claims that a guideline is being
fulfilled this may in fact be a false truth.
For example, if all images contain alt text then the software
will report a pass for this guideline. But what if the alt text
isn't descriptive of its image? What if alt text is crammed full of
nonsensical keywords for search engines? How can an automated
accessibility tool possibly know this?
Warnings may be misinterpreted
The reports generated by automated accessibility tools provide
warnings, as well as errors. These warnings are basically
guidelines that the automated tool can't check for, but which may
be errors. Often they're not, and in fact they're often not even
relevant. However, some people reading a report may try to get rid
of these warning messages by making the appropriate changes to
their site. By doing so, they may be implementing guidelines that
needn't be implemented and inadvertently lowering the website's
accessibility.
Conclusion
Automated accessibility testing tools can be useful as they can
save a large amount of time in performing some very basic checks
for accessibility. However, they must be used with caution and they
cannot be used as a stand-alone guide for accessibility checking.
Indeed, some expert accessibility knowledge should always be
applied in evaluating a site’s accessibility, perhaps in
conjunction with the fantastic web
accessibility toolbar to help dramatically speed up manual
checks.
© Webcredible 2005
