Score band . grade F
Sites scoring 30 to 39
53 of 2,613 fully measured sites land here, 2.0% of the index. Blocks most answer-surface crawlers, or serves almost nothing a crawler can read.
52 operators
One row per operator, highest score first. Partial assessments are excluded, because a renormalised score belongs to a different scale and could not be placed in a band honestly.
| Rank | Domain | Score | Policy | Stack |
|---|---|---|---|---|
| 2579 | elmundo.es | Score 39 out of 100, grade F | WalledDeliberate | Fastly |
| 4322 | kompas.com | Score 39 out of 100, grade F | SelectiveDeliberate | Amazon CloudFront |
| 3338 | orf.at | Score 39 out of 100, grade F | No trainingDeliberate | unidentified |
| 4005 | radiofrance.fr | Score 39 out of 100, grade F | SelectiveDeliberate | SvelteKit . Akamai |
| 1501 | xnxx.health | Score 39 out of 100, grade F | No trainingDeliberate | Cloudflare |
| 1817 | buzzfeed.com | Score 38 out of 100, grade F | SelectiveDeliberate | Next.js . Fastly |
| 2068 | filmyfly.bingo | Score 38 out of 100, grade F | No trainingDeliberate | Cloudflare |
| 914 | hbr.org | Score 38 out of 100, grade F | No trainingDeliberate | Next.js . Amazon CloudFront |
| 1539 | shahvani.com | Score 38 out of 100, grade F | No trainingDeliberate | Cloudflare |
| 630 | telegraph.co.uk | Score 37 out of 100, grade F | SelectiveDeliberate | Adobe Experience Manager . Fastly |
| 3358 | toyota.com | Score 37 out of 100, grade F | Assistant onlyDeliberate | Adobe Experience Manager . Amazon CloudFront |
| 1292 | actu.fr | Score 36 out of 100, grade F | SelectiveDeliberate | Cloudflare |
| 2921 | chicagotribune.com | Score 36 out of 100, grade F | WalledDeliberate | WordPress |
| 1767 | hdhub4u.bi | Score 36 out of 100, grade F | No trainingDeliberate | Cloudflare |
| 2078 | moviezwap.best | Score 36 out of 100, grade F | No trainingDeliberate | Cloudflare |
| 3763 | nydailynews.com | Score 36 out of 100, grade F | WalledDeliberate | WordPress |
| 1508 | pussyspace.com | Score 36 out of 100, grade F | No trainingDeliberate | Cloudflare |
| 4275 | 20minutes.fr | Score 35 out of 100, grade F | SelectiveDeliberate | Cloudflare |
| 2959 | asahi.com | Score 35 out of 100, grade F | WalledDeliberate | Akamai |
| 1063 | bfmtv.com | Score 35 out of 100, grade F | WalledDeliberate | Amazon CloudFront |
| 734 | dailymail.co.ukand 1 more domain with the same policy | Score 35 out of 100, grade F | WalledDeliberate | Akamai |
| 3444 | lifehacker.com | Score 35 out of 100, grade F | SelectiveDeliberate | Cloudflare |
| 2216 | mashable.com | Score 35 out of 100, grade F | SelectiveDeliberate | Cloudflare |
| 4595 | p4455.com | Score 35 out of 100, grade F | No trainingDeliberate | WordPress . Cloudflare |
| 2558 | pcmag.com | Score 35 out of 100, grade F | SelectiveDeliberate | Cloudflare |
| 4681 | test-uz.ru | Score 35 out of 100, grade F | SelectiveDeliberate | unidentified |
| 242 | bbc.co.uk | Score 34 out of 100, grade F | SelectiveDeliberate | Fastly |
| 955 | corriere.it | Score 34 out of 100, grade F | WalledDeliberate | Fastly |
| 2030 | sportfilm800.com | Score 34 out of 100, grade F | No trainingDeliberate | Next.js . Cloudflare |
| 4009 | thestar.com | Score 34 out of 100, grade F | WalledDeliberate | unidentified |
| 3965 | 2osb.com | Score 33 out of 100, grade F | No trainingDeliberate | Cloudflare |
| 2091 | blacktoon421.com | Score 33 out of 100, grade F | No trainingDeliberate | Cloudflare |
| 1789 | blacktoon423.com | Score 33 out of 100, grade F | No trainingDeliberate | Cloudflare |
| 1645 | dogdrip.net | Score 33 out of 100, grade F | No trainingDeliberate | Cloudflare |
| 2093 | filmyzilla65.com | Score 33 out of 100, grade F | No trainingDeliberate | Cloudflare |
| 2094 | filmyzilla67.com | Score 33 out of 100, grade F | No trainingDeliberate | Cloudflare |
| 2086 | filmyzilla68.com | Score 33 out of 100, grade F | No trainingDeliberate | Cloudflare |
| 1225 | gazzetta.it | Score 32 out of 100, grade F | WalledDeliberate | unidentified |
| 2667 | mastodon.social | Score 32 out of 100, grade F | No trainingDeliberate | Fastly |
| 3228 | metro.co.uk | Score 32 out of 100, grade F | WalledDeliberate | WordPress |
| 3401 | today.com | Score 32 out of 100, grade F | WalledDeliberate | Next.js . Akamai |
| 219 | bbc.com | Score 31 out of 100, grade F | SelectiveDeliberate | Next.js . Fastly |
| 4238 | channel4.com | Score 31 out of 100, grade F | SelectiveDeliberate | Akamai |
| 192 | cnn.com | Score 31 out of 100, grade F | SelectiveDeliberate | Fastly |
| 1361 | dagbladet.no | Score 31 out of 100, grade F | WalledDeliberate | Cloudflare |
| 4798 | dr.dk | Score 31 out of 100, grade F | WalledDeliberate | Next.js |
| 1977 | mercadolibre.com | Score 31 out of 100, grade F | SelectiveDeliberate | unidentified |
| 5013 | windstream.net | Score 31 out of 100, grade F | SelectiveDeliberate | Varnish |
| 4875 | yr.no | Score 31 out of 100, grade F | SelectiveDeliberate | unidentified |
| 4823 | band.us | Score 30 out of 100, grade F | Assistant onlyDeliberate | unidentified |
| 882 | nbcnews.com | Score 30 out of 100, grade F | WalledDeliberate | Next.js . Akamai |
| 2021 | videeyss.shop | Score 30 out of 100, grade F | No trainingDeliberate | Cloudflare |
Showing the first 52 operators. The full list is in the dataset.
crawl 2026-11-22 10:38 UTCprobe 3.0.0rubric 2.0.0registry 1.0.0vantage gha-ubuntu3,656 of 5,006 reachable
Using these figures
Free to reuse in research, journalism or a product under CC BY 4.0, with credit to Fidget Labs BV. Quote the measurement date so the claim stays checkable as the index moves.
CrawlIndex by Fidget Labs BV. "Sites scoring 30 to 39 for AI readiness." https://crawlindex.org (measured 2026-11-22). Licensed CC BY 4.0.