Automated alt text checks have a blind spot

WebAIM’s 2026 Million report across the top million home pages found 16.2% of images are missing alt text entirely. Of those that do include it, another 10.8% carry something useless — alt="image", a raw filename, or text copied from a neighboring image. That means more than one in four images on popular pages is described poorly or not at all.

The gap between fixing missing alt text and fixing bad alt text is where automated tooling struggles. Standard checkers verify that an accessible name exists; they rarely verify that the name says something meaningful about the image. That’s by design — a quality rule that generates false positives gets disabled. So alt="IMG_2847.png" passes, and so does an identical alt="3/5 stars" repeated across five star icons.

GitHub’s accessibility team built an alt text plugin for the GitHub Accessibility Scanner to close some of that gap. The rest of this piece covers the line they drew between provable defects and suspected ones, why layout mattered more than DOM order, and what changed when they brought a vision model into the check.

The checks that don’t need to see the picture

Whether an image is missing its alt attribute is objective. Whether a description is good is not. But between those extremes sits a set of checks that operate purely on the string — no image content needed:

  • The attribute is absent, empty, or whitespace-only.
  • The alt text is a filename like hero.png or IMG_2847.jpg.
  • The alt text is a placeholder such as TODO or tbd.
  • The alt text names the medium rather than the subject — image, logo, chart.
  • The same alt text appears across adjacent images.

Each of these is a claim about a string, and that became the dividing line for the plugin’s rule set. Five deterministic rules run by default and require no AI credentials or network calls. One opt-in rule uses a model with the image and page context to make calls that need more than the alt string alone.

Image selection leans on Playwright’s role-based locator rather than a raw querySelectorAll('img'). Anything excluded from the browser’s accessibility tree drops out — including images carrying alt="", which is the author explicitly marking an image decorative. Flagging that would punish correct behavior.

Strictness matters for adoption. The vague-alt rule normalizes a string and checks it against a curated list of words that carry no information. It only fires on exact matches:

  • alt="image" gets flagged.
  • alt="image of the login screen with the SSO button highlighted" doesn’t.

Literal rules miss plenty of weak alt text. The project accepts those misses to keep false positives near zero, reasoning that a checker teams keep enabled beats one they switch off.

Repeated text is a layout issue

Five star-shaped icons all reading "3/5 stars" is a real problem — a screen reader user hears the same phrase five times. The first version of the repetition check walked images in document order and flagged any run of identical normalized alt text. That produced bad findings: a footer “GitHub” logo and a header “GitHub” logo might be adjacent in the DOM order but nowhere near each other on screen, so no user experiences them as a run.

The rule now consults page layout. A run only extends when the gap between bounding boxes is small relative to the boxes themselves:

const gap = Math.max(horizontalGap, verticalGap) 
const largerDim = Math.max(a.boundingBox.width, a.boundingBox.height, 
                           b.boundingBox.width, b.boundingBox.height) 
return gap > GAP_MULTIPLIER * largerDim

Two implementation notes matter. The gap multiplier is a tuning judgment, not a value derived from a spec. And when an image has no measurable bounding box, the check fails open — the run continues. A missing finding is invisible; a wrong one isn’t.

Bringing a model into the loop

The deterministic rules need only the alt string. Anything smarter needs page context, which an <img> element on its own doesn’t carry. Whether alt="a smiling person" is sufficient depends on the page. Under a heading naming a specific person, it isn’t.

The optional alt-text-quality check gathers context with each image: the nearest heading, the page title, any <figcaption>, whether the image sits inside a link or button, and up to 600 characters of surrounding prose.

Link context is the most important signal. An image that is a link’s only content has its alt text become the link’s accessible name — so that alt should describe the destination, not the picture. One caveat: the plugin records only that the image is inside a link, not that it is the link’s sole content. Both cases currently look identical to the model.

The combined inputs — context, alt text, and image — go to a vision model through GitHub Models. The failure modes were rarely the model misreading an image; they were the model having opinions. Presented with perfectly good alt text, the first version would suggest alternatives, because “could this be better?” is a question a language model answers yes to on principle. Every image became a finding.

Three fixes stabilized it:

  • A structured decision procedure. The prompt walks four ordered steps, stops at the first match, and emits that step’s verdict: decorative, redundant with a caption, functional, or informative.
  • Explicit anti-nitpick rules. Redundant prefixes like “Image of…” are separated from semantic ones like “Photograph of….” A short alt is treated as correct when the surrounding prose already analyzes the image.
  • Forced field ordering in structured output. reasoning generates before verdict, so the model builds an argument before committing to a label.

None of this makes the model reliably correct. It makes it consistent enough to iterate against. The repository carries an offline grading harness built from published teaching material — WebAIM, the W3C images tutorial, and POET. The rule and harness share one prompt, so what gets tuned offline is what runs in CI. The harness only tests model judgment, not the full pipeline — a case can score perfectly there and still never reach the model in a real scan.

Privacy, cost, and the external model boundary

Once a check sends webpage data to an external model, it stops being a simple lint rule. Several consequences follow from that:

  • The rule is off by default. It runs only if deliberately enabled in the plugin configuration, and it needs a token with GitHub Models access.
  • URLs are redacted. Image URLs and link href values frequently carry signed CDN tokens or session identifiers. Query strings and fragments are stripped from anything entering the model context or the rule’s error logs. For the same reason, src and srcset in the markup sent to the model are replaced with (omitted).
  • Everything in that context window is untrusted input. Titles, headings, and prose come from the scanned page, and a page can contain text written to steer a model. Structured output limits the shape of a response, not the reasoning behind it.

It’s easy to over-read that list. Findings still carry the real page URL and original source into the scanner’s reporting pipeline — deliberately, since you can’t fix an image you can’t locate. Redaction limits what reaches the model and the logs, not what lands in your own issues. And an optional OCR pre-pass using Azure AI Vision credentials sends image bytes to a second destination. Azure isn’t required, but a data-flow review needs to account for both paths.

Cost follows the same shape: in the most common case, one model call per image per scan. On an image-heavy site, that can dominate the full run’s cost — reason enough to schedule it rather than run it on every commit.

What the plugin still can’t do

  • The deterministic rules are literal. They catch alt text that’s obviously unwritten, not fluent and wrong. They read the alt attribute rather than the computed accessible name, so an aria-label that fixes a problem won’t suppress the finding.
  • The model-backed rule produces false positives. Each finding is a prompt for human review, not a final verdict.
  • Silence is not coverage. The model-backed rule re-fetches images outside the browser session, so anything behind authentication can fail to load. Fetch and model errors are logged and skipped, which can leave a page looking clean when nothing was actually checked.
  • Suggested alt text is a draft. A model that sees an image and nearby words can’t account for your audience, style guide, or the job that image does on the page.
  • Findings can overlap with the scanner’s built-in checks. The plugin’s missing-alt rule covers ground the base scanner already handles.
  • Only HTML <img> tags are covered. SVG elements, role="img" containers, CSS backgrounds, and canvas aren’t checked yet.
  • This is new code with limited field feedback. Real-world markup and content variety is what improves rules like these, and the plugin hasn’t had that exposure. Treat early findings accordingly.
  • Passing isn’t conformance. Automated checks are a floor. Testing with people who use assistive technology is the goal.

Where the boundary sits

Separate what a checker can prove from what it can only suspect, and give those different defaults. Provable checks should be cheap, deterministic, and on by default. Suspicion-based checks belong behind an opt-in flag, framed as suggestions rather than verdicts.

The harder pattern is asking what a user experiences rather than what the DOM states. Every remaining gap in this plugin has that shape: it records an image is inside a link, not that the image is the link. It reads an attribute, not the computed accessible name.

That distance is the real boundary, and a better model doesn’t close it. Determining what an image does for a user who can’t see it still takes human judgment. Automation’s job is to make sure that human attention goes to the right images in the first place.