crawlindex

Cross-tab

AI readiness by publishing platform

What a site is built on shapes how legible it is to an agent, and often decides its crawler policy by default. Grouped from platform fingerprints detected on the homepage.

The spread is wide. wordpress sites block an answer-surface crawler 17.3% of the time. drupal sites, 1.8%.

AI readiness by publishing platform
PlatformSitesBlocking AIProportion blockingMean score
WordPress30052 (17.3%)72.3
Next.js28043 (15.4%)68.4
Adobe Experience Manager1306 (4.6%)68.6
Drupal1142 (1.8%)66.1
HubSpot CMS803 (3.8%)74.6
Nuxt574 (7.0%)64.3
Contentful501 (2.0%)70.6
Sanity303 (10.0%)78.7

Platforms are identified from response headers and markup, never guessed. An unrecognised stack is excluded rather than labelled. Cohorts smaller than 25 measured sites are not shown: a blocking rate over a handful of domains is noise presented as a finding.

Using these figures

Free to reuse in research, journalism or a product under CC BY 4.0, with credit to Fidget Labs BV. Quote the measurement date so the claim stays checkable as the index moves.

CrawlIndex by Fidget Labs BV. "AI readiness by publishing platform." https://crawlindex.org (measured 2026-08-09). Licensed CC BY 4.0.