Automated accessibility scans are tools for practitioners who understand their specific role and capabilities. Most consumers buying these tools don’t realize they’re purchasing something that won’t help them reach WCAG conformance.
In fact, many organizations buy scan-based platform subscriptions, not knowing the platform’s tracking is based entirely on scan results which means all of the data is skewed and somewhat meaningless (because we need to track full WCAG conformance).
Table of Contents
Only 13% of WCAG 2.2 AA Success Criteria Reliably Flagged
Automated scans can reliably flag just 7 of 55 WCAG 2.2 AA success criteria with mostly accurate detection. These are the technical, measurable criteria where mathematical calculations or binary checks produce consistent results—color contrast ratios, missing page titles, and HTML validation errors.
The remaining 87% of criteria either get partially flagged with varying reliability (45%) or cannot be detected at all (42%). Even the “reliably flagged” 13% require human review to verify context and implementation.
Consider color contrast detection, often cited as automation’s strength. Scans accurately measure contrast ratios for standard text and backgrounds, calculating whether white text on light gray meets the required 4.5:1 ratio. But they miss contrast issues when text appears over background images, gradients create varying contrast levels, or CSS pseudo-elements add overlays.
This fundamental limitation shapes everything about automated scanning. The tools excel at finding what’s missing or measuring what’s present, but they cannot evaluate meaning, context, or user experience. A scan will flag a missing alt attribute every time, but if your image has alt=”photo”, the scan sees the attribute exists and moves on—even though “photo” provides no useful information to someone using a screen reader.
For Practitioners, Not General Consumers
Accessibility professionals who understand these limitations use scans effectively. They know flagged issues are starting points for investigation. They recognize that fixing flagged issues without understanding context might not improve actual accessibility. They look beyond scan reports to find the 42% of issues that cannot be detected automatically.
The problem emerges when general consumers—organizations seeking WCAG conformance for compliance—purchase scan-based platforms under the false impression they’re essential for accessibility. These consumers use scans believing they’re working toward full WCAG conformance when they’re actually just improving their scan score.
Harvard University’s own accessibility documentation states: “While automated tools are a huge help in accessibility testing, they can’t catch every error since they can’t understand context or evaluate content quality. After you’ve done a first pass with the automated tools, you’ll need to follow up with manual testing to ensure that your website is inclusive and accessible.”
If you need the audit anyway, what are you buying the software for? The answer is clear for practitioners who use scans to quickly assess technical baselines, identify systemic issues, or monitor for regression. But for consumers whose objective is WCAG conformance, scans become an expensive detour.
Cannot Be Combined with “Manual Testing” to Create an Audit
Many accessibility vendors promote the idea that automated scans and manual testing must be combined to reach WCAG conformance. This creates a false equivalency that places automated scans and comprehensive audits on the same level.
We don’t combine automated scan results with accessibility issues identified through manual evaluation like two puzzle pieces. The automated scan results and issues identified through manual evaluation are not equal—scan results are wholly inferior.
An accessibility audit involves multiple evaluation methodologies: screen reader testing, keyboard testing, visual inspection, audible inspection, and code inspection. These comprehensive evaluations cover 100% of WCAG success criteria. When auditors use a scan, it’s as a secondary review layer to ensure any correctly flagged issues are included in the report.
Think in terms of layers, not combination. The first layer is the full technical evaluation where your digital asset is graded against WCAG using various methodologies. The second layer uses a scan as a review step. Manual evaluation swallows scan results whole—every relevant scan result is already included in the audit.
An audit report can be complete without a scan. Scan results are never complete. Using the word “combine” suggests we copy and paste scan results to complete our audit—when that is never the case.
There’s Virtually No Difference in Premium and Paid Scan Results
The AXE scan is free to use and reliably flags issues to the extent they can be flagged. Google Lighthouse, WAVE by WebAIM, and other free tools provide the same core detection capabilities as enterprise platforms costing thousands per month.
Enterprise tools often provide better reporting, integration features, and workflow management. They can scan pages requiring authentication, integrate with Jira, slice and dice results by WCAG success criteria, and generate analytics reports. But their core detection capabilities remain fundamentally limited by the same technical constraints.
Whether free or premium, all automated scanners face the same limitation—they cannot evaluate meaning, quality, or user experience. The price difference reflects convenience features and support rather than more comprehensive issue detection.
AI-enhanced scans have begun performing more sophisticated analyses than traditional rule-based scanning. They can evaluate alt text quality using natural language processing, identify potentially confusing content patterns, and suggest more descriptive link text. But AI scanning still cannot determine if content truly serves user needs.
These improvements make scans more useful for practitioners who understand their limitations, but AI scanning introduces new risks. While traditional scans are deterministic and consistent, AI scans can hallucinate problems that don’t exist or confidently suggest incorrect fixes. This unpredictability means AI scans require even more careful human verification than traditional scans.
Most Accessibility Platforms Are Scan-Based
The accessibility software market is projected to hit $800 million in 2025. Most of this market consists of scan-based platforms where the entire system—all analytics, progress reports, and data visualizations—relies on scan results.
Organizations using these platforms base their entire accessibility project around a scan, sometimes knowingly, often unknowingly. They strive to make their websites WCAG 2.1 AA or WCAG 2.2 AA conformant while simultaneously targeting a 100% score within the platform.
This disconnect between objective and measurement creates a fundamental problem. A scan might return a “100%” score because no technical flags were triggered, while an audit report identifies over 100 accessibility issues. The scan checked what it could measure; the audit evaluated everything.
This market reality is why audit-based platforms are necessary. When you upload your audit report—the result of a fully manual evaluation by a technical accessibility expert—you’re working from complete and accurate data. You know exactly which issues need fixing to reach WCAG conformance, not just which issues a scan can detect.
Key Insights
Automated accessibility scans serve practitioners who understand their role as rapid assessment tools for technical patterns. For consumers seeking WCAG conformance, scans represent an unnecessary expense and delay since comprehensive audits are required regardless.
The fundamental limitation—scans can only reliably flag 13% of WCAG criteria—remains consistent whether using free tools or enterprise platforms, with or without AI enhancement. The marketed idea of combining automated and manual testing misrepresents how accessibility evaluation actually works, where audits encompass all scan findings plus the 87% of criteria scans miss.
Most accessibility platforms being scan-based means organizations often track progress against incomplete data, working toward scan scores rather than actual WCAG conformance.
Frequently Asked Questions
Why do different scanning tools produce different results for the same website?
Each scanning tool uses different detection rules, algorithms, and heuristics. Some tools are more aggressive in flagging potential issues while others are more conservative. None can detect all WCAG issues, so variations reflect different approaches to the same fundamental limitations.
Can combining automated scans with screen reader testing provide complete WCAG coverage?
No, combining screen reader testing with a scan remains insufficient. Issues like color contrast, captions, small touch targets, complex gestures, and confusing instructions all pass screen reader testing while failing WCAG. Screen readers can also read past some technical issues that violate WCAG requirements.
If I fix all issues flagged by multiple different scanning tools, will my site be WCAG conformant?
No. Even if you run every available scan and fix every flagged issue, you’ll still miss approximately 42% of WCAG criteria that cannot be detected automatically. These include captions, audio descriptions, consistent navigation, error suggestions, and other criteria requiring human judgment.
Should organizations avoid using automated scans entirely?
Scans have value for practitioners who understand their limitations. Developers can incorporate scans into workflows to catch obvious issues during development. The problem arises when organizations purchase scan-based platforms believing they’re sufficient for WCAG conformance rather than starting with a comprehensive audit.
What’s the risk of an organization relying solely on automated scanning?
The risk is significant. Organizations believe they’re making progress toward WCAG conformance when they’re only addressing the small percentage of issues scans can detect. This false confidence can lead to compliance failures, as WCAG 2.1 AA conformance is either required or a best practice for many laws and regulations concerning digital accessibility.