Score band . grade F
Sites scoring 30 to 39
33 of 2,567 fully measured sites land here, 1.3% of the index. Blocks most answer-surface crawlers, or serves almost nothing a crawler can read.
All 33
One row per operator, highest score first. Partial assessments are excluded, because a renormalised score belongs to a different scale and could not be placed in a band honestly.
| Rank | Domain | Score | Policy | Stack |
|---|---|---|---|---|
| 4444 | radiofrance.fr | Score 39 out of 100, grade F | SelectiveDeliberate | SvelteKit . Akamai |
| 2479 | vidylee.clickpolicy gap | Score 38 out of 100, grade F | UndeclaredAbsent | Cloudflare |
| 2044 | yr.no | Score 38 out of 100, grade F | SelectiveDeliberate | unidentified |
| 122 | amazonvideo.com | Score 37 out of 100, grade F | SelectiveDeliberate | Amazon CloudFront |
| 236 | bbc.co.uk | Score 37 out of 100, grade F | SelectiveDeliberate | Fastly |
| 673 | primevideo.com | Score 37 out of 100, grade F | SelectiveDeliberate | Amazon CloudFront |
| 621 | telegraph.co.uk | Score 37 out of 100, grade F | SelectiveDeliberate | Adobe Experience Manager . Fastly |
| 3856 | toyota.com | Score 37 out of 100, grade F | Assistant onlyDeliberate | Adobe Experience Manager . Amazon CloudFront |
| 2922 | chicagotribune.com | Score 36 out of 100, grade F | WalledDeliberate | WordPress |
| 3785 | nydailynews.com | Score 36 out of 100, grade F | WalledDeliberate | WordPress |
| 3028 | thehindu.com | Score 36 out of 100, grade F | SelectiveDeliberate | Cloudflare |
| 4501 | 20minutes.fr | Score 35 out of 100, grade F | SelectiveDeliberate | Cloudflare |
| 3007 | asahi.com | Score 35 out of 100, grade F | WalledDeliberate | unidentified |
| 1083 | bfmtv.com | Score 35 out of 100, grade F | WalledDeliberate | Amazon CloudFront |
| 3540 | lifehacker.com | Score 35 out of 100, grade F | SelectiveDeliberate | Cloudflare |
| 2068 | mashable.com | Score 35 out of 100, grade F | SelectiveDeliberate | Cloudflare |
| 2368 | mparticle.com | Score 35 out of 100, grade F | SelectiveDeliberate | Next.js . Vercel |
| 218 | bbc.com | Score 34 out of 100, grade F | SelectiveDeliberate | Next.js . Fastly |
| 4179 | thestar.com | Score 34 out of 100, grade F | WalledDeliberate | unidentified |
| 2021 | ign.com | Score 33 out of 100, grade F | SelectiveDeliberate | unidentified |
| 3300 | metro.co.uk | Score 32 out of 100, grade F | WalledDeliberate | WordPress |
| 1171 | pimpbunny.com | Score 32 out of 100, grade F | SelectiveDeliberate | Cloudflare |
| 3149 | today.com | Score 32 out of 100, grade F | WalledDeliberate | Next.js . Akamai |
| 4865 | channel4.com | Score 31 out of 100, grade F | SelectiveDeliberate | Akamai |
| 204 | cnn.com | Score 31 out of 100, grade F | SelectiveDeliberate | Fastly |
| 4956 | dr.dk | Score 31 out of 100, grade F | WalledDeliberate | Next.js |
| 1896 | mercadolibre.com | Score 31 out of 100, grade F | SelectiveDeliberate | Amazon CloudFront |
| 5002 | tds.net | Score 31 out of 100, grade F | SelectiveDeliberate | Varnish |
| 1905 | wikihow.com | Score 31 out of 100, grade F | SelectiveDeliberate | unidentified |
| 485 | cnbc.com | Score 30 out of 100, grade F | SelectiveDeliberate | Akamai |
| 3357 | infojobs.net | Score 30 out of 100, grade F | WalledDeliberate | Amazon CloudFront |
| 890 | nbcnews.com | Score 30 out of 100, grade F | WalledDeliberate | Next.js . Akamai |
| 2781 | xnhau.xxx | Score 30 out of 100, grade F | SelectiveDeliberate | Cloudflare |
crawl 2026-09-22 08:34 UTCprobe 3.0.0rubric 2.0.0registry 1.0.0vantage gha-ubuntu3,618 of 5,006 reachable
Using these figures
Free to reuse in research, journalism or a product under CC BY 4.0, with credit to Fidget Labs BV. Quote the measurement date so the claim stays checkable as the index moves.
CrawlIndex by Fidget Labs BV. "Sites scoring 30 to 39 for AI readiness." https://crawlindex.org (measured 2026-09-22). Licensed CC BY 4.0.