webmcp-tool

Empirical benchmark · live corpus

State of Agent Readiness: What Public Websites Actually Expose to AI Agents

Most claims about the “agentic web” conflate early-adopter demos with the everyday internet, or grade sites by anecdote. Every figure on this page is computed live from our public scan archive on rule set 2026-08-31 — strictly separating an uncurated Tranco top-web sample from declared WebMCP adopters and user-requested scans.

69,187 comparable scans · 802 distinct third-party domains · 2026-08-31 to 2026-11-22 · 107 operator-owned domains excluded

Six citable findings

Each finding states its exact cohort, numerator, and denominator alongside public reports you can verify yourself.

93.5% of mainstream top sites (158 of 169) score Grade D or F, with a median Agent Readiness Score of 36/100.

Across the uncurated Tranco HTML sample, zero sites earn Grade A (0/169) and only 2 (1.2%) earn Grade B (mean score 36.8/100). While many pages are partially readable, reliable task completion and machine protocols remain rare on the general web.

36/100 median

llms.txt adoption (24.3%) outpaces callable MCP endpoints (2.4%) and browser WebMCP tools (1.2%) roughly 10-to-1.

41 of 169 reachable Tranco sites now ship a valid structured /llms.txt (including www.cloudflare.com, www.forter.com, amplitude.com, sentry.io, wordpress.org). Yet only 5 (3%) answer Accept: text/markdown, 4 (2.4%) expose an unblocked MCP discovery card, and 2 (1.2%) register browser WebMCP tools (www.cloudflare.com, www.youtube.com). Websites are summarizing themselves for LLMs far faster than they are letting agents act.

24.3% vs 1.2%

Nearly half of top homepages (49.1%) lose content without JavaScript, and only 5.8% of HTML forms are fully machine-labelled.

33 of 169 Tranco homepages (19.5%) arrive effectively empty without script execution, and another 50 (29.6%) arrive incomplete (why raw HTML matters). Among the 69 homepages serving HTML input forms, only 4 (5.8%) pass form semantics (labels, names, types, and autocomplete), and 1 (1.4%) expose machine-readable validation state.

5.8% form pass

Even among declared WebMCP early adopters (n = 370), only 41.1% cleanly pass on current document.modelContext registration.

60 listed adopter domains (16.2%) score partial — either still calling deprecated navigator.modelContext (deprecated in Chrome 150) or loading a polyfill without an observable registration. Another 158 (42.7%) expose no registration in raw HTML or up to ten same-origin bundles (gated behind auth, lazy chunks, or removed), and 238 (64.3%) omit readOnlyHint safety annotations.

16.2% partial

22.3% of live-probed sites refuse real agent user agents at the HTTP/WAF layer — while early adopters accidentally block their own manifests in robots.txt.

In the Tranco baseline, only 8.9% (15/169) explicitly allow all twelve named AI crawlers in robots.txt. Across 202 organic user-requested scans where live agent user-agent requests ran, 45 (22.3%) blocked at least one agent at the HTTP/WAF layer (26 refused ≥3 of 4 agents). And in the adopter cohort, 17 sites published an MCP card and 20 published an API description only to disallow it in robots.txt.

22.3% WAF-blocked

Repeat measurements (30+ days apart) already show measurable churn among early WebMCP deployments.

Among 132 WebMCP adopter hosts measured at least 30 days apart on the same rule set, 12 sites that previously passed modelcontext-present no longer pass on their latest scan (while 9 newly passed). Over the same span across 154 Tranco hosts, 96 scores did not move at all while 6 sites added llms.txt.

12 churned

Benchmark your own URL

Where does your site sit between the mainstream median (36/100) and early adopters (63/100)?

Run the same 25-check evaluation against any public URL in ~20 seconds — no account required.

Free instant score in ~20s. No account, no email required. Works on any public homepage, product, or form URL.

Three cohorts, separated on purpose

Pooling all 802 third-party domains into a single headline average would make llms.txt look 50% adopted across the internet — an artefact of including a directory of WebMCP early adopters. Separating the three sampling frames shows the real gap between the mainstream web and early protocol adopters.

MetricMainstream web (Tranco HTML)Declared WebMCP adoptersOrganic user scans
Distinct hosts (n)169370260
Median score36 / 10063 / 10044 / 100
Mean score (min–max)36.8 (7–87)63.3 (10–97)44.8 (9–100)
Grade A (90–100)0 (0%)24 (6.5%)8 (3.1%)
Grade A or B (≥75, ready / usable)2 (1.2%)103 (27.8%)23 (8.8%)
Grade D or F (<55, fails key paths)158 (93.5%)108 (29.2%)202 (77.7%)
Fitness axis (mean earned %)52.9%75.7%61%
Adoption axis (mean earned %)4.2%39.5%10.3%
Hosts with extracted WebMCP tool names0509

All 25 checks: pass, partial, and failure rates

Conditional checks (such as form semantics on pages with no form, or payment rails on non-commerce sites) are excluded from that check's denominator n rather than counted as failures. Every check links to its exact rule in the scoring methodology.

CheckPillarTranco passTranco partialTranco failAdopters pass
robots.txt lets AI crawlers in
robots-ai-crawlers · 5 pts · n=169
Agent Access8.9%
15/169
76.9%
130/169
14.2%
24/169
33.8%
125/370
Agent user agents are actually served
agent-user-agents-served · 5 pts · n=10
Agent Access90%
9/10
10%
1/10
0%
0/10
93.3%
125/134
No wall in front of the content
no-blocking-interstitial · 4 pts · n=169
Agent Access91.7%
155/169
5.3%
9/169
3%
5/169
89.2%
330/370
Content survives without JavaScript
content-without-javascript · 6 pts · n=169
Agent Access50.9%
86/169
29.6%
50/169
19.5%
33/169
85.1%
315/370
Semantic landmarks and heading order
semantic-structure · 4 pts · n=169
Machine Readability25.4%
43/169
32.5%
55/169
42%
71/169
62.2%
230/370
Valid structured data
structured-data · 6 pts · n=169
Machine Readability40.2%
68/169
4.1%
7/169
55.6%
94/169
82.2%
304/370
Markdown content negotiation
markdown-negotiation · 4 pts · n=169
Machine Readability3%
5/169
0%
0/169
97%
164/169
48.4%
179/370
llms.txt source file
llms-txt · 3 pts · n=169
Machine Readability24.3%
41/169
3.6%
6/169
72.2%
122/169
76.5%
283/370
Sitemap and canonical URL
sitemap-and-canonical · 3 pts · n=169
Machine Readability37.3%
63/169
45%
76/169
17.8%
30/169
79.2%
293/370
A primary action is discoverable
primary-action-present · 5 pts · n=169
Actionability49.1%
83/169
7.7%
13/169
43.2%
73/169
47.6%
176/370
Forms an agent can fill
form-quality · 7 pts · n=69
Actionability5.8%
4/69
81.2%
56/69
13%
9/69
19.6%
31/158
Controls have accessible names
named-controls · 5 pts · n=128
Actionability50%
64/128
35.9%
46/128
14.1%
18/128
79.7%
255/320
No bot challenge on the conversion path
no-captcha-on-conversion · 5 pts · n=169
Actionability92.3%
156/169
3.6%
6/169
4.1%
7/169
89.2%
330/370
Errors an agent can read
machine-readable-errors · 3 pts · n=69
Actionability1.4%
1/69
29%
20/69
69.6%
48/69
5.1%
8/158
document.modelContext is in use
modelcontext-present · 6 pts · n=169
WebMCP Implementation1.2%
2/169
0%
0/169
98.8%
167/169
41.1%
152/370
Declarative tools on forms
declarative-forms · 4 pts · n=74
WebMCP Implementation0%
0/74
0%
0/74
100%
74/74
30.5%
51/167
Tool names and descriptions+5 bonus
tool-hygiene · +5 pts · n=169
WebMCP Implementation1.2%
2/169
0%
0/169
98.8%
167/169
56.8%
210/370
Typed input schemas+5 bonus
input-schema-typed · +5 pts · n=169
WebMCP Implementation1.2%
2/169
0%
0/169
98.8%
167/169
53.2%
197/370
Honest annotations on tools+5 bonus
tool-annotations · +5 pts · n=169
WebMCP Implementation0.6%
1/169
0.6%
1/169
98.8%
167/169
34.6%
128/370
An MCP endpoint an agent can reach
mcp-endpoint · 7 pts · n=169
Integration2.4%
4/169
1.2%
2/169
96.4%
163/169
32.7%
121/370
A machine-readable description of your API
api-description · 5 pts · n=169
Integration5.3%
9/169
0%
0/169
94.7%
160/169
39.5%
146/370
An agent card other agents can read
agent-card · 3 pts · n=169
Integration3%
5/169
0%
0/169
97%
164/169
27.8%
103/370
Agent-facing authentication
agent-auth · 4 pts · n=11
Identity & Commerce27.3%
3/11
9.1%
1/11
63.6%
7/11
47.9%
91/190
Agentic payment rails
agentic-payments · 3 pts · n=25
Identity & Commerce4%
1/25
0%
0/25
96%
24/25
25.1%
49/195
Bot identity and content signals
bot-identity · 3 pts · n=169
Identity & Commerce4.1%
7/169
24.9%
42/169
71%
120/169
40.3%
149/370

Methodology, sampling frames, and limitations

A benchmark is only as trustworthy as what it refuses to conflate. Four constraints govern how every number on this page is computed:

  • Stratified sampling frames: The mainstream baseline is drawn in rank order from Tranco list 74V4X (first 250 domains whose homepage serves HTML over HTTPS; 169 permitted our identified crawler via robots.txt). The adopter cohort comes from webmcp.com's public type=live directory with Tranco overlaps removed (n = 370). Organic scans (n = 260) are third-party URLs submitted by visitors.
  • Conflict-of-interest exclusion: webmcp-tool.com and 106 other operator-run domains are filtered out before any cohort statistic is computed.
  • No user-agent spoofing on uninvited crawls: Our background corpus collector identifies itself as webmcp-tool.com/1.0, obeys robots.txt, and never impersonates GPTBot or ClaudeBot against sites nobody asked us to test. Consequently, agent-user-agents-served is excluded on background rows and reported strictly from 202 interactive third-party scans.
  • Single-URL, unauthenticated boundary: Each crawl inspects the target URL's raw HTTP response, robots.txt, sitemap.xml, llms.txt, standard /.well-known/* discovery paths, and up to ten same-origin JS bundles. WebMCP tools registered only behind authentication or lazy post-login routes will not appear in an unauthenticated public scan. See how Cloudflare and Ora/Vercel grade the same corpus and browse individual implementations in the WebMCP directory.

How to cite this benchmark

You are welcome to quote any statistic on this page with attribution and a link to this URL. Machine clients requesting this same URL with Accept: text/markdown receive the live data table in clean Markdown.

Wübbe, G. (2026). State of Agent Readiness: Empirical Benchmark of the Public Web (802 domains, ruleset 2026-08-31). WebMCP Tool. https://webmcp-tool.com/research

Need hands-on help fixing Agent Readiness or shipping WebMCP?

We work with engineering and agency teams to remediate raw-HTML and WAF blockers, expose declarative and imperative WebMCP tools, and ship remote MCP endpoints.

Talk to our engineering teamBrowse the leaderboard