General Information

Accessibility testing, or evaluation, is a key part of an ICT accessibility program.  Testing and evaluating the ICT should occur in the beginning of the development process as well as during the different stages of development.  A common mistake is to only test for accessibility after most designing and development is complete.  That makes the job of ensuring accessibility cumbersome and costly.

Testing is necessary to effectively include accessibility in the procurement of ICT.  Each organization needs to determine when to do its own testing and when to accept the vendor’s claims of accessibility, depending on factors such as level of risk, resources available for testing, size of user base and others.  In many instances, independent tests are necessary, conducted either by internal staff experts or outside consultants.

Tools and Manual Testing

There are many accessibility testing and evaluation tools; they are software or web-based.  There is no one tool that meets all needs.  The size of the organization and amount of ICT to be tested, technical needs, expertise level of developers and many other factors affect the decision of which tool(s) to use.

Automatic testing tools can detect several types of accessibility errors but cannot detect all possible issues.  Some can only be found through manual testing, and some can be identified but can only be fully checked through a combination of automatic and manual testing.  For example, an accessibility tool can determine whether an alt-tag exists for an image, but a human being must decide whether the text provided adequately describes the image in the context that is intended.

Evaluation Methodology and Report Templates

Accessibility testing should be repeatable with predictable and consistent results.  An evaluation methodology will achieve these goals.  There are methodologies such as the federal Trusted Tester methodology that are publicly available and can be readily adopted. The Trusted Tester program of the Department of Homeland Security offers free online training and certification.  Many organizations develop their own testing methodology so that it is compatible with their own IT management operations.

Along with a standardized evaluation methodology, it is important to use an accessibility testing report template.  The accessibility testing report is a record of what specific tests were conducted and the results.  Some reports provide information about what specific standards are covered and how to fix the issue that has been identified.  Reports can be used by developers to assess their progress over time, as well as by procurement staff for assessing the accessibility of a potential acquisition.  Keeping a repository of test reports is a valuable practice, since it can prevent having to repeat a test that had been done at a prior time.

User Testing

In addition to using automatic tools and conducting manual testing, involving people with disabilities in user testing provides information about the accessibility of IT that can’t be gained otherwise.  Some IT applications can meet the technical requirements for accessibility but at the same time prove to be very frustrating or difficult for people with disabilities to use.  It is also very educational for developers without much disability experience to observe and learn from actual users about how different assistive technologies work and to better understand what is the difference between ineffective and effective accessibility designs.

Resources

The website of the Web Accessibility Initiative of the W3C includes an extensive section on Testing and Evaluation.  It includes a general overview, a basic initial checklist, a list of automatic tools and guidance for how to make a selection, testing methodologies, report templates and information about involving users.

For technicians, a slide presentation by Paul J. Adam of Deque Systems provides much detailed explanations and links about testing.

The ICT Accessibility Testing Symposium is an annual conference with two days of presentations and workshops about testing and evaluation.  This link provides the programs and proceedings for past conferences.