The Benefits Of Cognitive Inclusion In UX Research — Smashing Magazine

Why Cognitive Testers Find More Usability Issues

In the summer of 2024, I co-chaired a working group of expert researchers at Fable, where I am VP of Innovation. Our goal was to determine best practices for performing accessibility testing with people who have cognitive disabilities — an umbrella term for conditions that affect memory, focus, and/or learning. Cognitive disability is the most prevalent disability in the U.S. (13.9% per CDC data) and is increasing rapidly, according to a Yale study.

We defined four objectives: how to recruit and screen participants effectively, what research best practices work with this audience, whether those methods hold up in a real study, and how to document our findings for sharing. We built a screener for people self-identifying with memory, focus, or learning challenges and reviewed published studies involving cognitive testers to collect best practices.

After a pilot with 25 testers, we refined our approach and produced both a guide for running user interviews with cognitive testers and a survey to quantify their digital product experiences. Once the pilot concluded, I had a strong hunch this group would surface more usability insights than general population (gen pop) participants I had worked with previously. I decided to test that hypothesis.

Study Setup and Method

Working with partners at the University of California, Irvine — including Syed Fatiul Huq and Fable researchers Pranav Pidathala, Ali Brown, and Michael Fagan — I ran a joint study across three AI-generated prototype websites, each with different goals and content:

We used a single screener with questions about memory, focus, and learning, splitting participants into two groups based on self-identified cognitive challenges.

Cognitive disability includes neurodiversity. Neurodivergent is an umbrella term used to describe people whose brains process information and learn differently. It is most commonly used for people who have learning disabilities (e.g., Dyslexia), ADHD, and Autism.

Thirty user interviews were conducted, ten per website, with an even split between cognitive and gen pop participants for each. In every session, participants completed all tasks for one website during an online interview facilitated by a researcher. All participants finished with the Accessible Usability Scale (AUS), a free, Creative Commons-licensed 10-question survey for evaluating website and app usability.

Analysis: Counting Issues and Suggestions

I reviewed all recordings and transcripts, logging every time a participant raised a concern, asked how something worked, or expressed difficulty. I counted each as an issue. I also noted when a participant missed part of a task even if they didn’t notice it themselves, plus every suggestion for improvement they offered. Examples collected included: “Photo is too tall and requires a lot of scrolling to get to content,” “I get no feedback when I like or dislike a book,” and an observed miss of a required P.O. Box checkbox. Suggestions ranged from wanting protein comparisons in a table to moving a “More information” tab higher on the page.

Each issue or suggestion was counted once per participant, though repeated themes across participants were expected and treated as signals of universal challenges.

Headline Findings

Across all three websites, cognitive participants identified 197 usability issues versus 113 from gen pop participants, and made 93 suggestions compared with 54. That works out to cognitive participants surfacing 1.8 times more issues and suggestions than the general population.

Broken down by category, cognitive participants flagged more problems with content, buttons and links, icons or visual elements, and media. Navigation issues were a near tie at 45 versus 46.

Site-by-Site Results

AUS scores range from 0 to 100, with higher numbers indicating better perceived usability.

Strong Snacks had the simplest design and content of the three sites, showing the fewest issues and the highest median AUS scores. Cognitive participants found 3.4 more issues and made 2.2 more suggestions on average, while scoring the overall experience 13.7 points lower than gen pop participants did.

Turning Pages offered the most varied functionality with four tasks. Cognitive participants found 6 more issues and made 3.2 more suggestions on average here, scoring the experience 17.2 points lower than gen pop on average.

Crown & Comb was deliberately complex — its third task, locating the bridal package, was designed to be very difficult. Cognitive participants found an average of 7 more issues and made 2.4 more suggestions. Notably, they scored the experience 14.3 points higher than gen pop participants did.

That inverted scoring pattern deserves attention: cognitive participants rated Crown & Comb higher than Turning Pages, while gen pop rated the opposite — Turning Pages higher, Crown & Comb lower. A plausible explanation is that cognitive participants found more issues on Turning Pages, which influenced their perception of its usability more heavily than the gen pop group’s perceptions were affected by issues there.

Category-level differences back this up: cognitive participants found far more button and link problems on Turning Pages, and more icon and visual element issues on Crown & Comb. This suggests the interaction challenges on Turning Pages carried more weight than the visual problems on Crown & Comb.

Qualitative Differences Between Groups

Patterns emerged in the commentary style and focus of each group. Cognitive participants were more likely to flag issues with icons or visual elements, surfaced content problems more often, and gave richer qualitative explanations of why something was hard to find or confusing. Gen pop participants were less likely to flag conceptual or comprehension barriers and tended to give shorter feedback, often stopping once a task was done.

The difference shows starkly in direct quotes from two Crown & Comb sessions, where both participants gave the lowest usability scores within their groups.

Gen pop participant quote

“As soon as you have a name of a treatment and a little explanation and like the duration and the price, as soon as you click onto that, it should be that you can interact with that service straight away. And I feel like if you're seeing a service repeated on a page multiple times and you're still not able to select it, it's really, really frustrating. This feels not particularly engaging.”

Cognitive participant quote

“For example, like, the mental energy aspect of it, like, sometimes there's, like, okay, cookies, and then ads, pop-ups, or maybe the website or service has too many options to look through, and maybe I just want something that I already know. I have to go through a lot of stuff. It makes me, like, feel drained and less able to focus.”

I read these quotes as showing a deeper impact on the cognitive participant: the experience was not merely frustrating but left them feeling drained and less able to focus. The data supports the practical conclusion: including participants with cognitive disabilities in UX research yields more issues, more improvement suggestions, and a clearer picture of the real burden a design imposes on its users.

What Cognitive Participants Uncovered in Real-World Tests

Across three separate website evaluations, the inclusion of participants with cognitive disabilities consistently surfaced issues that standard usability testing would likely have missed. These were not niche accessibility problems — they were core usability and trust issues affecting a much broader audience, including younger users accustomed to short-form content, seniors experiencing natural cognitive decline, and busy adults juggling work and family demands.

Strong Snacks: Trust, Context, and Distraction

On the Strong Snacks site, cognitive participants flagged several content and layout concerns that directly affected their ability to engage with the material:

• They wanted links to source material, such as scientific journals, to feel confident in the content's credibility.
• Headlines lacked sufficient context to understand what a blog post was about before clicking.
• The label Add-ons was confusing and unclear.
• Recipes appearing inline disrupted the main article flow, rather than being placed in a clearly demarcated sidebar.
• Ads and continuous animations distracted them from reading and comprehension.

These findings align with general design best practices, but the cognitive group made the consequences concrete: such distractions and unclear information architecture don’t just annoy users — they can prevent reading and comprehension entirely for some populations.

Turning Pages: Confusion in the Purchase Flow

On the book retailer site, cognitive participants revealed subtle but costly interaction problems. They were confused by how the Add to book bag button functioned and unclear about where reviews and recommendations originated. These issues directly undermined their trust in the site’s reliability.

While all participants could find the book-matching feature with effort, the cognitive group articulated a deeper problem: the site’s interactions were not predictable or consistent. This lack of predictability decreased their confidence in completing a purchase. In a retail context, ambiguity in a purchase path — compounded across thousands of users — inevitably translates to abandoned carts and lost revenue.

Crown & Comb: When Usability Becomes an Accessibility Barrier

The salon booking site highlighted some of the most significant findings from the study. Cognitive participants raised concerns about:

• The phrase “subject to stylist consultation”, which seemed to carry hidden conditions.
• Similar service labels that made it unclear whether they were booking the same service or different ones.
• The date selection step appearing too late in the booking flow.
• Ambiguity around when and how payment would be collected.

General-population participants were more likely to “muddle through” these ambiguities and complete the task. But that tolerance has a cost. When users encounter friction, they are far more likely to abandon the site for a competitor that offers a clearer, more predictable experience. Loyalty is tied to experience, not just brand.

The study demonstrated that finding a bridal package was hard for everyone, but for cognitive participants, it became an outright barrier. When a task combines ambiguity, excessive decision-making, insufficient feedback, and high search effort, cognitive load becomes overwhelming — that is the precise line where user experience challenges tip into accessibility failures.

Practical Takeaways for UX Research Teams

• Include cognitive participants in standard user research, not just accessibility audits. They surface general usability issues around content clarity, navigation labels, and interaction design while revealing cognitive-load friction other users may silently tolerate.
• Recognize that cognitive issues span usability and accessibility. Tasks relying on memory, focus, or decision-making can shift from difficult to impossible as cognitive load rises. That shift is where usability problems become accessibility barriers.
• Go beyond task completion metrics. Ask participants about their subjective experience: how a task affected their energy and focus, how distractions impacted their concentration, and how easy or difficult the task felt.
• Start small. Even a few sessions with participants who have cognitive access needs will provide actionable insight into managing cognitive load for the broader user base. Build from there over time.

Why Cognitive Inclusion Is a Strategic On-Ramp

The case for investing in cognitive accessibility research is demographic as well as ethical. The U.S. Census Bureau projects that by 2060, one in four Americans will be 65 or older — a population segment that naturally experiences cognitive decline. Companies building for these increasingly complex user needs today will be better positioned as that shift accelerates.

Cognitive accessibility work also serves as a practical entry point for teams new to inclusive research. Participants with cognitive access needs surface the same types of usability issues that UX teams already know how to fix — this is not a mysterious discipline requiring unfamiliar testing methods. Addressing cognitive load, clarity, and predictability first lays a research foundation that makes later work with assistive technology users more approachable.

The volume of insight from cognitive sessions is striking: in this exploratory study, participants with cognitive disabilities identified and suggested 1.8 times more improvements than general-population participants. As one UX manager at Bell Media put it: “2 sessions with cognitive users feel like 200 because of the volume of insights we get.”

Cognitive inclusion in UX research is therefore not optional, nor is it purely an accessibility exercise. It is a way for UX teams to make their research more efficient, produce clearer content and simpler flows, and ultimately ship better products for everyone.

Study Limitations and Context

This was a small exploratory study, so findings are qualitative rather than statistically validated. Sessions were conducted on two different platforms — Fable Engage for cognitive participants and UserFeel for the general population — which may have influenced participant panels and comfort levels. The author disclosed working for Fable, noting the platform was selected partly for affordability to increase participant numbers. Different facilitators ran each group, though all used the same task structure and discussion guide; the author personally counted all issues and suggestions to maintain consistency across sites and participants.

Resources for Getting Started

• W3C’s supplemental guidance on cognitive accessibility and the more detailed material from the cognitive accessibility task force.
• MDN’s overview of how WCAG maps to cognitive accessibility supports.
• A case study of Fable’s cognitive accessibility pilot.
• Five simple design fixes for calmer digital spaces, benefiting neurodivergent users and everyone else.
• Neurodiversity and UX: essential resources for cognitive accessibility.