Score band . grade F
Sites scoring 20 to 29
18 of 2,567 fully measured sites land here, 0.7% of the index. Blocks most answer-surface crawlers, or serves almost nothing a crawler can read.
15 operators
One row per operator, highest score first. Partial assessments are excluded, because a renormalised score belongs to a different scale and could not be placed in a band honestly.
| Rank | Domain | Score | Policy | Stack |
|---|---|---|---|---|
| 2001 | nrk.no | Score 28 out of 100, grade F | SelectiveDeliberate | SvelteKit |
| 1315 | f95zone.to | Score 27 out of 100, grade F | SelectiveDeliberate | Cloudflare |
| 1939 | franceinfo.fr | Score 27 out of 100, grade F | WalledDeliberate | unidentified |
| 3668 | stuff.co.nz | Score 27 out of 100, grade F | SelectiveDeliberate | Fastly |
| 692 | dailymail.co.ukand 1 more domain with the same policy | Score 26 out of 100, grade F | WalledDeliberate | Akamai |
| 2902 | faz.net | Score 26 out of 100, grade F | SelectiveDeliberate | Nuxt |
| 1410 | iltalehti.fi | Score 26 out of 100, grade F | WalledDeliberate | Amazon CloudFront |
| 1113 | theconversation.com | Score 26 out of 100, grade F | WalledDeliberate | Next.js . Fastly |
| 1404 | dw.com | Score 24 out of 100, grade F | SelectiveDeliberate | Akamai |
| 3688 | politico.eu | Score 24 out of 100, grade F | WalledDeliberate | WordPress . Cloudflare |
| 1287 | themoviedb.org | Score 24 out of 100, grade F | WalledDeliberate | Amazon CloudFront |
| 2635 | tmdb.org | Score 24 out of 100, grade F | WalledDeliberate | Amazon CloudFront |
| 586 | usatoday.com | Score 24 out of 100, grade F | WalledDeliberate | unidentified |
| 647 | amazon.ca | Score 22 out of 100, grade F | SelectiveDeliberate | Amazon CloudFront |
| 277 | launchpad.net | Score 20 out of 100, grade F | SelectiveDeliberate | unidentified |
Showing the first 15 operators. The full list is in the dataset.
crawl 2026-09-22 08:34 UTCprobe 3.0.0rubric 2.0.0registry 1.0.0vantage gha-ubuntu3,618 of 5,006 reachable
Using these figures
Free to reuse in research, journalism or a product under CC BY 4.0, with credit to Fidget Labs BV. Quote the measurement date so the claim stays checkable as the index moves.
CrawlIndex by Fidget Labs BV. "Sites scoring 20 to 29 for AI readiness." https://crawlindex.org (measured 2026-09-22). Licensed CC BY 4.0.