Top-level domain
.org
226 measured sites. 31 of them, 13.7%, block at least one answer-surface AI crawler. Mean readiness score 65.2.
Compare against every other top-level domain
| Rank | Domain | Score | Answer-surface crawlers | Stack |
|---|---|---|---|---|
| 39 | wikipedia.org | Score 66 out of 100, grade C | all allowed | unidentified |
| 68 | wordpress.org | Score 89 out of 100, grade B | all allowed | WordPress |
| 86 | ntp.org | Score 55 out of 100, grade D | all allowed | unidentified |
| 93 | mozilla.org | Score 60 out of 100, grade C | all allowed | Fastly |
| 95 | nginx.org | Score 62 out of 100, grade C | all allowed | unidentified |
| 108 | lencr.org | Score 54 out of 100, grade D | all allowed | Netlify |
| 140 | ipify.org | Score 58 out of 100, grade D | all allowed | Amazon CloudFront |
| 141 | apache.org | Score 60 out of 100, grade C | all allowed | Fastly |
| 142 | duckdns.org | Score 54 out of 100, grade D | all allowed | unidentified |
| 147 | archive.org | Score 92 out of 100, grade A, partial assessment | all allowed | unidentified |
| 163 | telegram.org | Score 52 out of 100, grade D | all allowed | unidentified |
| 176 | creativecommons.org | Score 73 out of 100, grade C | all allowed | WordPress . Cloudflare |
| 186 | w3.org | Score 84 out of 100, grade B, partial assessment | all allowed | Cloudflare |
| 205 | doi.org | Score 54 out of 100, grade D | all allowed | Cloudflare |
| 217 | wikimedia.org | Score 54 out of 100, grade D | all allowed | unidentified |
| 326 | debian.org | Score 57 out of 100, grade D | all allowed | unidentified |
| 362 | gnu.orgrefused GPTBot | Score 54 out of 100, grade D | all allowed | unidentified |
| 374 | un.orgrefused GPTBot | Score 45 out of 100, grade D | all allowed | Drupal . Amazon CloudFront |
| 400 | cookiedatabase.orgrefused GPTBot | Score 78 out of 100, grade B, partial assessment | all allowed | Cloudflare |
| 453 | arxiv.org | Score 57 out of 100, grade D | all allowed | Fastly |
| 464 | featureassets.orgrefused GPTBot | Score 78 out of 100, grade B, partial assessment | all allowed | Google Cloud / GFE |
| 476 | dyndns.org | Score 52 out of 100, grade D | all allowed | unidentified |
| 488 | ietf.orgrefused GPTBot | Score 53 out of 100, grade D | all allowed | Cloudflare |
| 489 | openstreetmap.orgrefused GPTBot | Score 48 out of 100, grade D | all allowed | Fastly |
| 516 | npr.org | Score 60 out of 100, grade C | all allowed | Akamai |
| 603 | ieee.org | Score 78 out of 100, grade B, partial assessment | all allowed | Amazon CloudFront |
| 641 | iso.orgrefused GPTBot | Score 78 out of 100, grade B, partial assessment | all allowed | Cloudflare |
| 682 | cambridge.orgrefused GPTBot | Score 86 out of 100, grade B, partial assessment | 1 blocked | Cloudflare |
| 699 | rfc-editor.org | Score 64 out of 100, grade C | all allowed | Nuxt . Cloudflare |
| 730 | cookielaw.orgrefused GPTBot | Score 51 out of 100, grade D | 5 blocked | Adobe Experience Manager . Cloudflare |
| 737 | craigslist.org | Score 60 out of 100, grade C | all allowed | unidentified |
| 746 | python.org | Score 65 out of 100, grade C | all allowed | Fastly |
| 778 | ampproject.org | Score 57 out of 100, grade D | all allowed | Netlify |
| 847 | spamhaus.org | Score 60 out of 100, grade C | all allowed | Nuxt . Cloudflare |
| 861 | frontiersin.org | Score 83 out of 100, grade B | all allowed | Nuxt . Azure Front Door |
| 891 | worldbank.org | Score 82 out of 100, grade B | all allowed | Adobe Experience Manager . Cloudflare |
| 914 | hbr.orgrefused GPTBot | Score 38 out of 100, grade F | 5 blocked | Next.js . Amazon CloudFront |
| 920 | allaboutcookies.org | Score 78 out of 100, grade B, partial assessment | all allowed | Cloudflare |
| 923 | networkadvertising.org | Score 70 out of 100, grade C | all allowed | WordPress . Cloudflare |
| 942 | mayoclinic.org | Score 78 out of 100, grade B, partial assessment | all allowed | Akamai |
| 951 | pbs.org | Score 64 out of 100, grade C | all allowed | Next.js . Amazon CloudFront |
| 973 | change.org | Score 74 out of 100, grade C | all allowed | Amazon CloudFront |
| 996 | ecosia.orgrefused GPTBot | Score 84 out of 100, grade B, partial assessment | all allowed | Cloudflare |
| 1006 | oecd.orgrefused GPTBot | Score 92 out of 100, grade A, partial assessment | all allowed | Cloudflare |
| 1010 | themoviedb.orgrefused GPTBot | Score 18 out of 100, grade F, partial assessment | 11 blocked | Amazon CloudFront |
| 1019 | thenai.org | Score 70 out of 100, grade C | all allowed | WordPress . Cloudflare |
| 1047 | weforum.org | Score 78 out of 100, grade B, partial assessment | all allowed | Akamai |
| 1074 | archiveofourown.orgrefused GPTBot | Score 47 out of 100, grade D | 2 blocked | Cloudflare |
| 1097 | adblockplus.org | Score 67 out of 100, grade C | all allowed | WordPress . Fastly |
| 1119 | hltv.orgrefused GPTBot | Score 78 out of 100, grade B, partial assessment | all allowed | Cloudflare |
| 1122 | jw.org | Score 92 out of 100, grade A, partial assessment | all allowed | Amazon CloudFront |
| 1139 | plos.orgrefused GPTBot | Score 92 out of 100, grade A, partial assessment | all allowed | Cloudflare |
| 1140 | lichess.orgrefused GPTBot | Score 84 out of 100, grade B, partial assessment | all allowed | Cloudflare |
| 1147 | apa.org | Score 71 out of 100, grade C | all allowed | Imperva |
| 1172 | letsencrypt.org | Score 57 out of 100, grade D | all allowed | Netlify |
| 1175 | postgresql.org | Score 67 out of 100, grade C | all allowed | Varnish |
| 1236 | coursera.org | Score 80 out of 100, grade B | 1 blocked | Contentful . Amazon CloudFront |
| 1248 | jstor.org | Score 74 out of 100, grade C, partial assessment | 3 blocked | Fastly |
| 1333 | kernel.org | Score 54 out of 100, grade D | all allowed | unidentified |
| 1353 | wiktionary.org | Score 66 out of 100, grade C | all allowed | unidentified |
| 1374 | e-hentai.org | Score 62 out of 100, grade C | all allowed | Cloudflare |
| 1378 | tgju.org | Score 74 out of 100, grade C | all allowed | unidentified |
| 1381 | bbb.org | Score 85 out of 100, grade B | all allowed | Cloudflare |
| 1386 | pornhub.org | Score 69 out of 100, grade C | all allowed | unidentified |
| 1387 | aternos.org | Score 14 out of 100, grade F, partial assessment | 11 blocked | unidentified |
| 1403 | fedoraproject.orgrefused GPTBot | Score 60 out of 100, grade C | all allowed | Nuxt |
| 1406 | iana.org | Score 58 out of 100, grade D | all allowed | Cloudflare |
| 1510 | exhentai.org | Score 6 out of 100, grade F, partial assessment | 11 blocked | unidentified |
| 1544 | acm.orgrefused GPTBot | Score 92 out of 100, grade A, partial assessment | all allowed | Cloudflare |
| 1573 | zearn.org | Score 58 out of 100, grade D | all allowed | unidentified |
| 1602 | tjk.org | Score 6 out of 100, grade F, partial assessment | 11 blocked | unidentified |
| 1643 | syosetu.orgrefused GPTBot | Score 78 out of 100, grade B, partial assessment | all allowed | Cloudflare |
| 1644 | anybunny.org | Score 55 out of 100, grade D | all allowed | Cloudflare |
| 1656 | eff.org | Score 64 out of 100, grade C | all allowed | Drupal . Fastly |
| 1742 | newtoki1.orgrefused GPTBot | Score 78 out of 100, grade B, partial assessment | all allowed | Azure Front Door |
| 1754 | xxxpostpic.org | Score 77 out of 100, grade B | all allowed | WordPress . Cloudflare |
| 1782 | bpexch.orgrefused GPTBot | Score 28 out of 100, grade F | 5 blocked | Cloudflare |
| 1794 | fao.org | Score 70 out of 100, grade C | all allowed | Cloudflare |
| 1812 | acs.org | Score 67 out of 100, grade C | all allowed | Adobe Experience Manager . Fastly |
| 1845 | pewresearch.org | Score 88 out of 100, grade B | all allowed | WordPress |
| 1849 | moodle.orgrefused GPTBot | Score 88 out of 100, grade B, partial assessment | all allowed | Cloudflare |
| 1853 | orcid.org | Score 48 out of 100, grade D | all allowed | Cloudflare |
| 1936 | clevelandclinic.org | Score 73 out of 100, grade C | all allowed | Next.js . Cloudflare |
| 1938 | nosv.org | Score 58 out of 100, grade D | all allowed | unidentified |
| 1966 | afraid.org | Score 60 out of 100, grade C | all allowed | unidentified |
| 2022 | sciencemag.orgrefused GPTBot | Score 68 out of 100, grade C, partial assessment | 4 blocked | Cloudflare |
| 2031 | mitre.org | Score 78 out of 100, grade B, partial assessment | all allowed | unidentified |
| 2045 | freebsd.org | Score 57 out of 100, grade D | all allowed | unidentified |
| 2099 | pnas.orgrefused GPTBot | Score 86 out of 100, grade B, partial assessment | 1 blocked | Cloudflare |
| 2123 | collegeboard.org | Score 62 out of 100, grade C | all allowed | Drupal . Akamai |
| 2135 | bitbucket.org | Score 78 out of 100, grade B | all allowed | unidentified |
| 2166 | semanticscholar.orgrefused GPTBot | Score 54 out of 100, grade D | all allowed | HubSpot CMS . Amazon CloudFront |
| 2169 | science.orgrefused GPTBot | Score 68 out of 100, grade C, partial assessment | 4 blocked | Cloudflare |
| 2197 | unicef.org | Score 73 out of 100, grade C | all allowed | Drupal . Cloudflare |
| 2212 | altervista.org | Score 64 out of 100, grade C | all allowed | Amazon CloudFront |
| 2267 | nodejs.org | Score 73 out of 100, grade C | all allowed | Next.js . Cloudflare |
| 2273 | openssl.org | Score 6 out of 100, grade F, partial assessment | 11 blocked | unidentified |
| 2299 | wikisource.org | Score 66 out of 100, grade C | all allowed | unidentified |
| 2324 | signal.org | Score 57 out of 100, grade D | all allowed | Cloudflare |
| 2365 | icann.orgrefused GPTBot | Score 56 out of 100, grade D | all allowed | Angular . Cloudflare |
| 2367 | kde.org | Score 61 out of 100, grade C | all allowed | unidentified |
| 2370 | browserleaks.org | Score 58 out of 100, grade D | all allowed | unidentified |
| 2381 | aboutcookies.org | Score 71 out of 100, grade C | all allowed | Sitecore . Cloudflare |
| 2393 | khanacademy.org | Score 78 out of 100, grade B, partial assessment | all allowed | Varnish |
| 2410 | imf.org | Score 78 out of 100, grade B, partial assessment | all allowed | Akamai |
| 2474 | nejm.orgrefused GPTBot | Score 68 out of 100, grade C, partial assessment | 4 blocked | Cloudflare |
| 2490 | pypi.org | Score 64 out of 100, grade C | all allowed | Fastly |
| 2496 | opensource.orgrefused GPTBot | Score 92 out of 100, grade A, partial assessment | all allowed | Cloudflare |
| 2633 | tizen.orgrefused GPTBot | Score 78 out of 100, grade B, partial assessment | all allowed | Amazon CloudFront |
| 2652 | cve.org | Score 92 out of 100, grade A, partial assessment | all allowed | Amazon CloudFront |
| 2655 | tmdb.orgrefused GPTBot | Score 18 out of 100, grade F, partial assessment | 11 blocked | Amazon CloudFront |
| 2663 | matomo.org | Score 80 out of 100, grade B | all allowed | WordPress . Amazon CloudFront |
| 2670 | perl.org | Score 58 out of 100, grade D | all allowed | Fastly |
| 2688 | slashdot.org | Score 73 out of 100, grade C | all allowed | Cloudflare |
| 2707 | archlinux.org | Score 62 out of 100, grade C | all allowed | unidentified |
| 2715 | drupal.org | Score 84 out of 100, grade B, partial assessment | all allowed | Fastly |
| 2725 | apachefriends.org | Score 61 out of 100, grade C | all allowed | Fastly |
| 2785 | ilo.orgrefused GPTBot | Score 59 out of 100, grade D | all allowed | Drupal . Cloudflare |
| 2792 | oneidrive.org | Score 6 out of 100, grade F, partial assessment | 11 blocked | unidentified |
| 2793 | npmjs.orgrefused GPTBot | Score 78 out of 100, grade B, partial assessment | all allowed | Cloudflare |
| 2839 | centos.org | Score 57 out of 100, grade D | all allowed | unidentified |
| 2882 | iop.orgrefused GPTBot | Score 53 out of 100, grade D | all allowed | Drupal . Cloudflare |
| 2913 | iea.orgrefused GPTBot | Score 78 out of 100, grade B, partial assessment | all allowed | Cloudflare |
| 2934 | phys.org | Score 83 out of 100, grade B | all allowed | unidentified |
| 2948 | undp.org | Score 84 out of 100, grade B, partial assessment | all allowed | unidentified |
| 2975 | userway.org | Score 74 out of 100, grade C | all allowed | Gatsby . Amazon CloudFront |
| 2984 | rsc.org | Score 92 out of 100, grade A, partial assessment | all allowed | Cloudflare |
| 2996 | heart.org | Score 64 out of 100, grade C | all allowed | Sitecore . Cloudflare |
| 3035 | ohchr.orgrefused GPTBot | Score 78 out of 100, grade B, partial assessment | all allowed | Cloudflare |
| 3047 | ntppool.org | Score 55 out of 100, grade D | all allowed | Fastly |
| 3048 | drpc.org | Score 86 out of 100, grade B | all allowed | Next.js . Cloudflare |
| 3049 | aarp.orgrefused GPTBot | Score 60 out of 100, grade C | all allowed | Adobe Experience Manager . Amazon CloudFront |
| 3062 | owasp.org | Score 63 out of 100, grade C | all allowed | Next.js . Vercel |
| 3072 | freedesktop.org | Score 58 out of 100, grade D | all allowed | Fastly |
| 3096 | cpan.org | Score 60 out of 100, grade C | all allowed | Fastly |
| 3133 | imagemagick.org | Score 69 out of 100, grade C | all allowed | Fastly |
| 3134 | faqs.org | Score 52 out of 100, grade D | 1 blocked | unidentified |
| 3150 | wikibooks.org | Score 66 out of 100, grade C | all allowed | unidentified |
| 3173 | joomla.org | Score 66 out of 100, grade C | all allowed | Joomla . Cloudflare |
| 3177 | videolan.org | Score 67 out of 100, grade C | all allowed | unidentified |
| 3185 | openldap.org | Score 57 out of 100, grade D | all allowed | unidentified |
| 3215 | mediawiki.org | Score 74 out of 100, grade C | all allowed | unidentified |
| 3233 | unep.org | Score 65 out of 100, grade C | 5 blocked | Cloudflare |
| 3256 | nobelprize.orgrefused GPTBot | Score 63 out of 100, grade C | all allowed | WordPress . Cloudflare |
| 3260 | man7.org | Score 52 out of 100, grade D | all allowed | unidentified |
| 3271 | worldcat.orgrefused GPTBot | Score 48 out of 100, grade D | 3 blocked | Adobe Experience Manager . Cloudflare |
| 3314 | isc.org | Score 62 out of 100, grade C | all allowed | Fastly |
| 3318 | gnome.org | Score 57 out of 100, grade D | all allowed | Fastly |
| 3345 | wireshark.org | Score 60 out of 100, grade C | all allowed | Astro . Cloudflare |
| 3374 | geeksforgeeks.org | Score 67 out of 100, grade C | all allowed | WordPress . Amazon CloudFront |
| 3389 | zenodo.org | Score 66 out of 100, grade C | all allowed | unidentified |
| 3402 | codeberg.org | Score 31 out of 100, grade F, partial assessment | 8 blocked | unidentified |
| 3410 | hrw.org | Score 63 out of 100, grade C | all allowed | Drupal . Fastly |
| 3427 | cancer.org | Score 64 out of 100, grade C | all allowed | Adobe Experience Manager . Fastly |
| 3439 | greenpeace.org | Score 56 out of 100, grade D | all allowed | Cloudflare |
| 3440 | code.org | Score 74 out of 100, grade C | all allowed | Next.js . Amazon CloudFront |
| 3445 | consumerreports.orgrefused GPTBot | Score 54 out of 100, grade D | 3 blocked | Adobe Experience Manager . Amazon CloudFront |
| 3476 | familysearch.org | Score 58 out of 100, grade D, partial assessment | all allowed | Amazon CloudFront |
| 3486 | assetsconfigcdn.orgrefused GPTBot | Score 78 out of 100, grade B, partial assessment | all allowed | Google Cloud / GFE |
| 3632 | metmuseum.orgrefused GPTBot | Score 78 out of 100, grade B, partial assessment | all allowed | Vercel |
| 3639 | gimp.org | Score 60 out of 100, grade C | all allowed | Fastly |
| 3669 | hopkinsmedicine.orgrefused GPTBot | Score 78 out of 100, grade B, partial assessment | all allowed | Cloudflare |
| 3685 | amnesty.org | Score 73 out of 100, grade C | 1 blocked | WordPress . Cloudflare |
| 3698 | ourworldindata.org | Score 61 out of 100, grade C | all allowed | Cloudflare |
| 3706 | lua.org | Score 45 out of 100, grade D | all allowed | unidentified |
| 3743 | opensuse.orgrefused GPTBot | Score 48 out of 100, grade D | all allowed | unidentified |
| 3745 | edx.org | Score 74 out of 100, grade C | all allowed | Next.js . Cloudflare |
| 3797 | torproject.org | Score 57 out of 100, grade D | all allowed | unidentified |
| 3827 | mariadb.org | Score 80 out of 100, grade B | all allowed | WordPress |
| 3869 | ispconfig.org | Score 67 out of 100, grade C | all allowed | Cloudflare |
| 3888 | eurekalert.org | Score 89 out of 100, grade B, partial assessment | all allowed | unidentified |
| 3937 | nessus.org | Score 62 out of 100, grade C | all allowed | Drupal . Cloudflare |
| 3972 | unhcr.orgrefused GPTBot | Score 92 out of 100, grade A, partial assessment | all allowed | Cloudflare |
| 3978 | js.org | Score 54 out of 100, grade D | all allowed | Cloudflare |
| 3998 | wikidata.org | Score 66 out of 100, grade C | all allowed | unidentified |
| 4047 | blender.orgrefused GPTBot | Score 53 out of 100, grade D, partial assessment | 5 blocked | Cloudflare |
| 4064 | nber.org | Score 63 out of 100, grade C | all allowed | Drupal . Fastly |
| 4077 | propublica.org | Score 80 out of 100, grade B | all allowed | WordPress . Cloudflare |
| 4116 | chromium.org | Score 92 out of 100, grade A, partial assessment | all allowed | Fastly |
| 4136 | rfa.org | Score 68 out of 100, grade C | all allowed | Akamai |
| 4185 | redcross.org | Score 54 out of 100, grade D | all allowed | Adobe Experience Manager . Akamai |
| 4195 | purl.org | Score 58 out of 100, grade D | all allowed | unidentified |
| 4209 | wikiquote.org | Score 66 out of 100, grade C | all allowed | unidentified |
| 4221 | unwomen.orgrefused GPTBot | Score 78 out of 100, grade B, partial assessment | all allowed | Fastly |
| 4227 | emojipedia.orgrefused GPTBot | Score 53 out of 100, grade D | 4 blocked | Ghost . Cloudflare |
| 4287 | raspberrypi.org | Score 69 out of 100, grade C | all allowed | Contentful . Cloudflare |
| 4288 | ghost.org | Score 67 out of 100, grade C | all allowed | Netlify |
| 4295 | squid-cache.org | Score 58 out of 100, grade D | all allowed | unidentified |
| 4297 | nationalacademies.org | Score 84 out of 100, grade B, partial assessment | all allowed | Akamai |
| 4313 | tvtropes.orgrefused GPTBot | Score 54 out of 100, grade D, partial assessment | 6 blocked | Cloudflare |
| 4321 | doxygen.org | Score 58 out of 100, grade D | all allowed | Cloudflare |
| 4327 | readthedocs.orgrefused GPTBot | Score 53 out of 100, grade D | all allowed | Cloudflare |
| 4333 | unicode.org | Score 84 out of 100, grade B, partial assessment | all allowed | Cloudflare |
| 4362 | eu.org | Score 58 out of 100, grade D | all allowed | unidentified |
| 4367 | ahajournals.orgrefused GPTBot | Score 57 out of 100, grade D, partial assessment | 5 blocked | Cloudflare |
| 4375 | biorxiv.orgrefused GPTBot | Score 78 out of 100, grade B, partial assessment | all allowed | Cloudflare |
| 4409 | eclipse.org | Score 57 out of 100, grade D | all allowed | unidentified |
| 4411 | openoffice.org | Score 58 out of 100, grade D | all allowed | unidentified |
| 4415 | libpng.org | Score 55 out of 100, grade D | all allowed | unidentified |
| 4424 | virtualbox.org | Score 55 out of 100, grade D | all allowed | Akamai |
Showing the 200 most-visited.
Using these figures
Free to reuse in research, journalism or a product under CC BY 4.0, with credit to Fidget Labs BV. Quote the measurement date so the claim stays checkable as the index moves.
CrawlIndex by Fidget Labs BV. "Agent readiness on .org." https://crawlindex.org (measured 2026-11-22). Licensed CC BY 4.0.