Testing Sites And Apps With Blind Users: A Cheat Sheet — Smashing Magazine

Why Automated Checks Aren’t Enough

Plugins and services for checking accessibility are useful, but they can’t replace observing a real person with a disability use your site or app. Meeting accessibility checklists is essential, but compliance doesn’t always translate into a pleasant experience. Live testing sessions provide insights that automated reports simply cannot capture.

This article focuses on users of screen readers — software that converts source code into speech. While typically used by people with low vision or blindness, screen readers serve others too. These users will surface the majority of your accessibility issues. The topic is broad, but the following guide should help you get started.

Audits vs. Testing

Evaluating the accessibility of a digital product can be approached in two distinct ways.

Auditing compares a site or app element-by-element against a set of requirements, such as universal standards like WCAG or country-specific laws like ADA in the U.S. or AODA in Ontario. Audits come in two forms:

1. Automated audit
   Checking accessibility with web apps, design and coding plugins, or browser extensions like axe DevTools, ARC Toolkit, WAVE, or Stark. These tools produce a report of issues and recommendations.
2. Expert audit
   Evaluation by a professional with deep knowledge of accessibility requirements. This person may use assistive technology or have a disability themselves, but they are an expert, not a “common user.” The resulting report is more contextual and sensible than an automated one.

Testing cannot be done by one person alone. It involves users of assistive technologies in a series of one-on-one sessions facilitated by a designer, UX researcher, or other professional. While auditing checks compliance, testing reveals how real users experience your product.

Usability vs. Accessibility Testing

Usability testing is a well-known research method. Accessibility testing shares several traits but has key distinctions.

Common features:

• Script
  A facilitator prepares a full written script with introduction, questions, and realistic tasks (e.g., buying a ticket or ordering a taxi). Testing script templates are readily available.
• Insights gathering
  Accessibility testing reveals usability issues as well. A facilitator should ask follow-up questions to understand the participant’s thinking, pain points, and needs.
• Format
  Both can be conducted online or offline. Sessions typically run 30 to 60 minutes.

Key differences:

• Participant selection
  Usability testing recruits by demographics such as job title, gender, or country. Accessibility testing considers the senses and assistive technologies involved in using the product.
• What you can test
  Usability testing works with live products, interactive prototypes (in Figma, Protopie, Framer, etc.), or static mockups. Accessibility testing generally requires a live product since prototyping tools can’t deliver source code that assistive technology can parse. Figma has attempted accessible prototypes, but it’s still far from perfect.
• Giving hints
  When usability participants get stuck, you guide them. With disabled users, you must understand how their assistive gear works. A phrase like “click on the red cross icon in the corner” is meaningless to a blind user.

The Case for Testing

Testing offers two major advantages over auditing alone:

1. Valuable insights.
   Testing shows whether the entire flow works and whether people can reach their goals. Unlike audits, it’s grounded in real assistive technology usage by a person with a disability.
2. Empathy through storytelling.
   A compelling story is more persuasive than bare numbers. Even one or two thorough sessions give you enough material to excite your team about accessibility in a way an audit report might not.

Testing yields more realistic insights into everyday scenarios. Laws and standards aren’t perfect, and formal compliance doesn’t always cover real user challenges. People often take the path that feels safer or more intuitive rather than the designed one — and testing reveals that.

Audits remain a powerful method, but combining them with testing produces far more accurate results.

Recruiting the Right Users

Disabilities vary widely, and so do the assistive technologies that support web browsing. Here’s a quick overview:

• By senses or affected areas: visual (blindness, color deficiency, low vision), physical (cerebral palsy, amputation, arthritis), cognitive (dyslexia, Down syndrome, autism), and auditory (deafness, hearing loss).
• By severity: permanent (e.g., an amputated leg or innate condition), temporary (e.g., a broken arm or blurred vision after eye drops), and situational (e.g., a noisy room or carrying a child).

Visit the Microsoft Inclusive Design hub for more on disability types.

For most digital products that primarily rely on vision, several visual assistive technologies offer alternative ways to access content:

• Screen readers convert text into speech with shortcuts for efficient navigation.
• Refreshable Braille displays show lines of tactile Braille text with round-tipped pins that rise and refresh as the user moves their cursor. These are essential for blind-deaf users.
• Virtual assistants (Amazon Alexa, Apple Siri, Google Assistant) interpret speech and respond with synthesized voices — a universal design example serving both disabled and non-disabled users.
• High-contrast displays or modes for people with low vision. Some users pair high-contrast mode with a screen reader.

Who to Involve

The debate over ideal participant numbers is endless, but for a first-time accessibility test, the following guidance applies:

• Invite 3–6 users with blindness or low vision who rely on screen readers or special modes like extra zoom or increased contrast.
• If your product features rich data visualization (charts, dashboards, maps), involve several people with color blindness as well.

Even one or two high-quality sessions are better than a dozen poorly prepared ones.

Where to Find Participants

Finding testers isn’t as difficult as it seems. For mass-market products, participants only need proficiency with their assistive technology. Three reliable sources:

• Specialized platforms like Access Works or UserTesting allow you to recruit by specific parameters. This is the fastest route, though not the cheapest — platforms add their commission on top of user compensation.
• Social media communities for people with disabilities. Search for keywords like “people with disabilities,” “PWD,” “support group,” “visually impaired,” or “blind people.” Ask admin permission before posting your announcement.
• Social enterprises and non-profits in inclusion, employment, and disability support — e.g., Inclusive IT in Ukraine or The Federation of the Blind and Partially Sighted in Germany. Email them with your request.

The last two options may sound like free recruiting, but not everyone can volunteer. In our own university-course testing, three participants joined pro bono. Otherwise, plan to compensate participants (roughly €15–30 in our experience) with gift cards or useful coupons — just ensure the compensation is itself accessible.

Companies that test accessibility regularly often hire people with disabilities so they can iteratively check in-progress software before launch.

Setting Up for the Session

The first decision is whether to test remotely or face-to-face. Remote sessions let participants work in their own environment with familiar assistive technology, remove travel costs, and widen the pool of potential candidates. Face-to-face testing makes sense for products that aren't yet public, for observing how people interact with mobile devices, and for sessions where participants may need hands-on help getting comfortable with their assistive technology.

If you go remote, resist the urge to adopt a specialized usability testing platform. Tools like UserTesting, Lookback, or UserZoom offer features like transcription and dashboards, but they're expensive, and participants may find them unfamiliar. A standard video conferencing tool with screen-sharing and call recording covers what you actually need, and most people will already know how to use it. Whichever tool you pick, work out how to explain screen-sharing in non-visual terms before the session begins.

Tasks That Could Happen in Real Life

Accessibility testing only works with a product that is at least in alpha or beta. You want to observe real behavior, not have people imagine how they might act. That means writing tasks people can relate to: describe a scenario rather than point at interface elements.

Begin with a short interview to learn about the participant's background. If you're testing an air travel service, ask whether they fly often and where they like to go, then use their answers to shape a task about booking a flight to a destination they actually care about.

Good, realistic task:

• "You want to buy a gift card for your colleague George who enjoys bikepacking. Choose the card value, customize other preferences, and select how George will receive the gift."

Task that leads the participant too heavily:

• "Open the main menu and find the 'Other' category. Choose a €50 gift card. In the 'For whom' input field enter 'John Doe'… Select 'Visa/Mastercard' as a paying method…"

A session is equal parts preparation and conversation. When someone gets stuck, hints should guide them toward exploring possibilities, not tell them where to click. Something like:

— No worries. So, the search doesn't give the expected result. What else can you do here?
— Hmm, I don't know. Maybe filtering it somehow…
— OK, please try that.

Choosing Your Words Carefully

Language affects how participants think and how much bias creeps into their feedback. Watch out for a few common pitfalls:

• Leading instructions like "Go to the Dashboard section and find the frequency chart" reveal exactly what you expect to happen and destroy any chance of observing natural behavior.
• Salesy phrasing such as "Try the Smart filtering feature" makes people feel obligated to praise the product.
• Jokes in tasks ("Request Christmas tree delivery to Lapland") pull people out of the scenario you're trying to create.
• Technical interface terms like "toggle switch" or "dropdown" confuse participants who don't use that vocabulary, and often signal that you're drifting into leading hints.

Running the Session

Your first accessibility session will likely involve someone using a screen reader, so it helps to know what that software does.

A screen reader converts visual page content into speech. It ignores style-related markup like colors and fonts, and instead works from structural elements: heading tags, image descriptions, and labels on buttons, inputs, and checkboxes. Well-structured code is easier to interpret, and participants have an easier time making sense of the page.

Every mainstream operating system includes a built-in screen reader: VoiceOver on Mac and iOS, Narrator on Windows, and TalkBack on Android. Real-world users often prefer third-party options, particularly JAWS (paid, Windows) and NVDA (free, Windows).

Giving Helping Hands

Screen reader users navigate with a keyboard or touchscreen, keeping one element in focus at a time rather than scanning a whole page visually.

At some point, your participant will hit an obstacle, usually an element that isn't keyboard-navigable or lacks a proper label. When that happens, you'll need hints that don't rely on visual description.

Hints that don't work:

• "Click the cross icon in the upper right corner."
• "Scroll to the bottom of the modal window and find the button there."
• "Look at the table in the center of the page."

Hints that do:

• "Please, navigate to the next/previous item."
• "Go to the second element in the list."
• "Select the last heading/link/button."

Turning Session Output Into Action

After the sessions wrap up, sift through the collected feedback, rank it by priority, and draft an action plan. Three principles will keep this phase honest:

• Log, don’t rely on memory. Sessions generate a flood of observations, and human recall is fallible. Don’t depend solely on recordings. Take notes as you go, or have an assistant do it. Notes are far easier to scan for recurring patterns across participants, and they guard against a failed recording wiping out your data.
• Raw data ≠ insight. What you observe is not always what it seems. A participant who uses search instead of filters may simply find typing less effortful than navigating a filter menu. Distinguish the observable event from the underlying reason, motivation, or mental model that drives it.
• Weight by criticality and impact. Not every observation warrants a fix. If five users can’t reach the shopping cart because it isn’t keyboard-navigable, that’s a hard barrier for both users and revenue. But one participant disliking a button label is low stakes. Judge each finding by how many participants hit the problem and how much it blocks their core goal — booking a ticket, ordering pizza, or sending a document.

Once you’ve organized the findings, take them back to the whole team — designers, engineers, product managers, QA. The more interactive you make the readout, the better. Let colleagues dig into the data, ask questions, and map results to their own responsibilities. With practice, you can invite teammates to watch a session live (Google Meet works fine) or project it to a quiet room of observers, with the hard rule that they stay silent and out of the participant’s way.

Digging Deeper

• “How to Conduct Usability Studies for Accessibility,” Nielsen Norman Group
• “How to Incorporate Users with Disabilities in UX Testing,” Deque Systems
• “Tips for Conducting Usability Studies with Participants with Disabilities,” Peter McNally, Smashing Magazine
• “Accessibility in User-Centered Design: Planning Usability Testing,” from Just Ask: Integrating Accessibility Throughout Design, Shawn Lawton Henry
• “Conducting Mobile Accessibility Research with Screen-Reader Users,” Tanner Kohler, Nielsen Norman Group