crawlindex

Search the index. A domain that is not here can still be measured live on the check page.

Score band . grade F

Sites scoring 20 to 29

18 of 2,613 fully measured sites land here, 0.7% of the index. Blocks most answer-surface crawlers, or serves almost nothing a crawler can read.

17 operators

One row per operator, highest score first. Partial assessments are excluded, because a renormalised score belongs to a different scale and could not be placed in a band honestly.

Sites scoring 20 to 29
RankDomainScorePolicyStack
2148ign.comScore 29 out of 100, grade FSelectiveDeliberateWordPress . Fastly
1479joyclub.deScore 29 out of 100, grade FWalledDeliberateCloudflare
1050pimpbunny.comScore 29 out of 100, grade FSelectiveDeliberateunidentified
1753bpexch.live
and 1 more domain with the same policy
Score 28 out of 100, grade FNo trainingDeliberateCloudflare
1447ekstrabladet.dkScore 28 out of 100, grade FWalledDeliberateFastly
4085nrk.noScore 28 out of 100, grade FSelectiveDeliberateSvelteKit
3273franceinfo.frScore 27 out of 100, grade FWalledDeliberateunidentified
4744stuff.co.nzScore 27 out of 100, grade FSelectiveDeliberateFastly
2772faz.netScore 26 out of 100, grade FSelectiveDeliberateNuxt
1407iltalehti.fiScore 26 out of 100, grade FWalledDeliberateAmazon CloudFront
1195theconversation.comScore 26 out of 100, grade FWalledDeliberateNext.js . Fastly
486cnbc.comScore 24 out of 100, grade FWalledDeliberateAkamai
1402dw.comScore 24 out of 100, grade FSelectiveDeliberateAkamai
3574politico.euScore 24 out of 100, grade FWalledDeliberateWordPress . Cloudflare
1612wnacg.comScore 24 out of 100, grade FAssistant onlyDeliberateCloudflare
4601gentoo.orgScore 21 out of 100, grade FWalledDeliberateFastly
296launchpad.netScore 20 out of 100, grade FSelectiveDeliberateunidentified

Showing the first 17 operators. The full list is in the dataset.

crawl 2026-11-22 10:38 UTCprobe 3.0.0rubric 2.0.0registry 1.0.0vantage gha-ubuntu3,656 of 5,006 reachable

Using these figures

Free to reuse in research, journalism or a product under CC BY 4.0, with credit to Fidget Labs BV. Quote the measurement date so the claim stays checkable as the index moves.

CrawlIndex by Fidget Labs BV. "Sites scoring 20 to 29 for AI readiness." https://crawlindex.org (measured 2026-11-22). Licensed CC BY 4.0.