Help and advice on user testing for accessibility?

Hi all!

I work at a small web development agency up in Portland, ME and I’m preparing to run a usability test on a web application we’ve built with a participant using a screenreader. Our development team just made a bunch of updates to the site to move it closer to ADA (Americans with Disabilities Act) compliance, so we’re trying to find out if our first pass actually improved the site’s accessibility and what work still needs to be done.

The problem is: I’ve never run a usability test with a participant using a screenreader. I have basic experience running usability tests, so I have an ok handle on how to moderate a test session, but I want to learn the basics of testing the user-friendliness of web accessibility features so I at least sort of know what I’m doing here :slight_smile:

Specifically, I’m wondering:

  • Do you have any advice on how to test the usability of a site’s accessibility features?
  • What adaptations, if any, should I think to make to my typical usability test setup?
    • The participant and I will be connecting remotely, I’m hoping over video call, with him sharing his screen. I have no idea if this’ll work or if asking him to navigate through a video conferencing app (Google Hangouts) could complicate the test unnecessarily.
  • Should I provide the participant instructions for using the site or can I leave them in the dark, let them figure out the site on their own?
    • For a typical usability test, I’d want the participant to know as little as possible about the site under test, but I don’t know if omitting usage instructions — included dev team’s accessibility work — would prevent the user from even interacting with the site.

Sorry if my questions show my ignorance of web accessibility i.e. anything sounds goofy or dumb. I’m totally new to the topic.


Hi zackbrs,

If your test will be run on a Windows machine, you can install NVDA to listen to the site yourself, before the test. It’s available for anyone to use, no charge. If you are running the test on a Mac, use Window Eyes.

Is your representative user who is logging in from home a visually impaired user who regularly uses screen readers? Will he be working with his own screen reader remotely?

Any adaptations to your normal testing routine will be dependent on your measurable objectives of the accessibility test.

Just present the participant with the URL of the site, with no instructions other than your task list. You’ll want an unbiased case study.

So the compliance you’ll need to work with is WCAG. The ADA is mostly about building compliance, such as “ADA seating” or proper wheelchair ramps. When the ADA refers to web site accessibility, they refer developers over to WCAG.