Top-level domain

.org

Name: CrawlIndex dataset
License: https://creativecommons.org/licenses/by/4.0/

215 measured sites. 23 of them, 10.7%, block at least one answer-surface AI crawler. Mean readiness score 66.4.

Compare against every other top-level domain

Sites on .org
Rank	Domain	Score	Answer-surface crawlers	Stack
31	wikipedia.org	Score 67 out of 100, grade C	all allowed	unidentified
43	wordpress.org	Score 95 out of 100, grade A	all allowed	WordPress
70	ntp.org	Score 56 out of 100, grade D	all allowed	unidentified
79	nginx.org	Score 64 out of 100, grade C	all allowed	unidentified
92	mozilla.org	Score 62 out of 100, grade C	all allowed	Fastly
122	apache.org	Score 62 out of 100, grade C	all allowed	Fastly
127	cdn77.org	Not scored	all allowed	unidentified
133	prodregistryv2.org	Not scored	all allowed	unidentified
145	archive.org	Score 53 out of 100, grade D	all allowed	unidentified
181	w3.org	Score 89 out of 100, grade B, partial assessment	all allowed	Cloudflare
197	creativecommons.org	Score 75 out of 100, grade B	all allowed	WordPress . Cloudflare
203	doi.org	Score 56 out of 100, grade D	all allowed	Amazon CloudFront
224	lencr.org	Score 53 out of 100, grade D, partial assessment	all allowed	Netlify
244	telegram.org	Score 53 out of 100, grade D	all allowed	unidentified
247	wikimedia.org	Score 53 out of 100, grade D	all allowed	unidentified
285	duckdns.org	Score 55 out of 100, grade D	all allowed	unidentified
293	debian.org	Score 53 out of 100, grade D	all allowed	unidentified
323	gnu.org	Not scored	all allowed	unidentified
331	un.orgrefused GPTBot	Score 46 out of 100, grade D	all allowed	Amazon CloudFront
385	cookiedatabase.orgrefused GPTBot	Score 83 out of 100, grade B, partial assessment	all allowed	Cloudflare
386	ampproject.org	Score 60 out of 100, grade C	all allowed	Netlify
393	cookielaw.orgrefused GPTBot	Score 55 out of 100, grade D	5 blocked	Adobe Experience Manager . Cloudflare
462	ietf.orgrefused GPTBot	Score 55 out of 100, grade D	all allowed	Cloudflare
468	featureassets.orgrefused GPTBot	Score 83 out of 100, grade B, partial assessment	all allowed	Google Cloud / GFE
479	npr.org	Score 56 out of 100, grade D	all allowed	Akamai
482	arxiv.org	Score 59 out of 100, grade D	all allowed	Fastly
499	beyondwickedmapping.org	Not scored	all allowed	unidentified
580	ieee.org	Score 48 out of 100, grade D	all allowed	Amazon CloudFront
585	ipify.org	Score 59 out of 100, grade D	all allowed	HubSpot CMS . Amazon CloudFront
616	unesco.org	Score 78 out of 100, grade B	all allowed	Drupal
643	dyndns.org	Score 53 out of 100, grade D	all allowed	unidentified
660	cambridge.orgrefused GPTBot	Score 94 out of 100, grade A, partial assessment	1 blocked	Cloudflare
674	openstreetmap.orgrefused GPTBot	Score 49 out of 100, grade D	all allowed	Fastly
716	allaboutcookies.org	Score 83 out of 100, grade B, partial assessment	all allowed	Cloudflare
722	iso.orgrefused GPTBot	Score 83 out of 100, grade B, partial assessment	all allowed	Cloudflare
741	python.org	Score 63 out of 100, grade C	all allowed	Fastly
757	networkadvertising.org	Score 72 out of 100, grade C	all allowed	WordPress . Cloudflare
805	hbr.org	Score 53 out of 100, grade D	3 blocked	Next.js . Amazon CloudFront
833	thenai.org	Score 72 out of 100, grade C	all allowed	WordPress . Cloudflare
843	mayoclinic.org	Score 83 out of 100, grade B, partial assessment	all allowed	Akamai
864	afilias-nst.org	Not scored	all allowed	unidentified
879	worldbank.org	Score 80 out of 100, grade B	all allowed	Adobe Experience Manager . Cloudflare
887	frontiersin.org	Score 91 out of 100, grade A	all allowed	Nuxt . Azure Front Door
901	change.org	Score 79 out of 100, grade B	all allowed	Amazon CloudFront
935	weforum.org	Score 83 out of 100, grade B, partial assessment	all allowed	Akamai
939	ci-servers.org	Not scored	all allowed	unidentified
992	pbs.org	Not scored	all allowed	unidentified
1029	ecosia.orgrefused GPTBot	Score 89 out of 100, grade B, partial assessment	all allowed	Cloudflare
1062	oecd.orgrefused GPTBot	Score 100 out of 100, grade A, partial assessment	all allowed	Cloudflare
1073	adblockplus.org	Score 56 out of 100, grade D	all allowed	Fastly
1096	apa.orgrefused GPTBot	Score 69 out of 100, grade C	all allowed	Imperva
1108	coursera.org	Score 84 out of 100, grade B	1 blocked	Contentful . Amazon CloudFront
1116	letsencrypt.org	Score 59 out of 100, grade D	all allowed	Netlify
1128	archiveofourown.orgrefused GPTBot	Score 46 out of 100, grade D	2 blocked	Cloudflare
1134	hltv.orgrefused GPTBot	Score 83 out of 100, grade B, partial assessment	all allowed	Cloudflare
1183	jw.org	Score 64 out of 100, grade C	all allowed	Amazon CloudFront
1225	lichess.org	Score 59 out of 100, grade D	all allowed	unidentified
1234	jstor.org	Score 81 out of 100, grade B, partial assessment	3 blocked	Fastly
1252	kernel.org	Score 56 out of 100, grade D	all allowed	unidentified
1271	postgresql.org	Score 67 out of 100, grade C	all allowed	Varnish
1280	wiktionary.org	Score 67 out of 100, grade C	all allowed	unidentified
1325	aternos.orgrefused GPTBot	Score 89 out of 100, grade B, partial assessment	all allowed	Cloudflare
1379	pornhub.org	Score 73 out of 100, grade C	all allowed	unidentified
1380	e-hentai.org	Score 64 out of 100, grade C	all allowed	Cloudflare
1422	iana.org	Score 59 out of 100, grade D	all allowed	Cloudflare
1459	nosv.org	Score 59 out of 100, grade D	all allowed	unidentified
1462	eff.org	Score 56 out of 100, grade D	all allowed	Drupal . Fastly
1476	fedoraproject.orgrefused GPTBot	Score 63 out of 100, grade C	all allowed	Nuxt
1517	exhentai.org	Score 7 out of 100, grade F, partial assessment	11 blocked	unidentified
1550	acm.orgrefused GPTBot	Score 100 out of 100, grade A, partial assessment	all allowed	Cloudflare
1595	search-start.org	Not scored	all allowed	unidentified
1600	syosetu.org	Score 56 out of 100, grade D	2 blocked	Cloudflare
1636	fao.org	Score 71 out of 100, grade C	all allowed	Cloudflare
1645	tjk.org	Score 53 out of 100, grade D	all allowed	unidentified
1745	ganamosnet.orgrefused GPTBot	Score 83 out of 100, grade B, partial assessment	all allowed	Cloudflare
1779	bbb.org	Score 67 out of 100, grade C	all allowed	Cloudflare
1785	acs.org	Score 67 out of 100, grade C	all allowed	Adobe Experience Manager . Fastly
1796	pewresearch.org	Score 92 out of 100, grade A	all allowed	WordPress
1818	moodle.org	Score 62 out of 100, grade C	all allowed	WordPress . Cloudflare
1859	tgju.org	Score 76 out of 100, grade B	all allowed	Cloudflare
1879	sciencemag.orgrefused GPTBot	Score 75 out of 100, grade B, partial assessment	4 blocked	Cloudflare
1898	clevelandclinic.org	Score 79 out of 100, grade B	all allowed	Cloudflare
1945	xxxpostpic.org	Score 83 out of 100, grade B	all allowed	WordPress . Cloudflare
2010	spinnovax.org	Not scored	all allowed	unidentified
2041	orcid.org	Score 48 out of 100, grade D	all allowed	Cloudflare
2046	mitre.org	Score 83 out of 100, grade B, partial assessment	all allowed	unidentified
2061	rfc-editor.org	Score 67 out of 100, grade C	all allowed	Nuxt . Cloudflare
2066	spamhaus.org	Score 64 out of 100, grade C	all allowed	Nuxt . Cloudflare
2074	science.orgrefused GPTBot	Score 75 out of 100, grade B, partial assessment	4 blocked	Cloudflare
2088	freebsd.org	Score 63 out of 100, grade C	all allowed	unidentified
2094	themoviedb.org	Score 27 out of 100, grade F	11 blocked	Amazon CloudFront
2128	unicef.orgrefused GPTBot	Score 83 out of 100, grade B, partial assessment	all allowed	Cloudflare
2131	bitbucket.orgrefused GPTBot	Score 71 out of 100, grade C	all allowed	Amazon CloudFront
2136	aboutcookies.org	Score 75 out of 100, grade B	all allowed	Sitecore . Cloudflare
2165	plos.org	Score 83 out of 100, grade B	all allowed	HubSpot CMS . Cloudflare
2266	nodejs.org	Score 79 out of 100, grade B	all allowed	Next.js . Cloudflare
2274	craigslist.org	Score 63 out of 100, grade C	all allowed	unidentified
2275	kde.org	Score 64 out of 100, grade C	all allowed	unidentified
2324	semanticscholar.orgrefused GPTBot	Score 57 out of 100, grade D	all allowed	HubSpot CMS . Amazon CloudFront
2358	icann.orgrefused GPTBot	Score 89 out of 100, grade B, partial assessment	all allowed	Cloudflare
2360	openssl.org	Score 7 out of 100, grade F, partial assessment	11 blocked	unidentified
2366	signal.org	Score 59 out of 100, grade D	all allowed	Cloudflare
2374	apachefriends.org	Score 64 out of 100, grade C	all allowed	Fastly
2409	privacy-center.org	Not scored	all allowed	unidentified
2418	nejm.orgrefused GPTBot	Score 75 out of 100, grade B, partial assessment	4 blocked	Cloudflare
2455	altervista.org	Score 67 out of 100, grade C	all allowed	Amazon CloudFront
2462	imf.org	Score 100 out of 100, grade A, partial assessment	all allowed	Akamai
2568	slashdot.org	Score 76 out of 100, grade B	all allowed	Cloudflare
2577	afraid.org	Score 59 out of 100, grade D	all allowed	unidentified
2585	cve.org	Score 56 out of 100, grade D	all allowed	Amazon CloudFront
2596	drupal.org	Score 52 out of 100, grade D	all allowed	Fastly
2612	khanacademy.org	Score 49 out of 100, grade D	all allowed	Varnish
2633	tizen.orgrefused GPTBot	Score 83 out of 100, grade B, partial assessment	all allowed	Amazon CloudFront
2641	perl.org	Score 59 out of 100, grade D	all allowed	Fastly
2652	gutenberg.org	Score 59 out of 100, grade D	all allowed	unidentified
2671	matomo.org	Score 83 out of 100, grade B	all allowed	WordPress . Amazon CloudFront
2678	pnas.orgrefused GPTBot	Score 94 out of 100, grade A, partial assessment	1 blocked	Cloudflare
2679	tmdb.org	Score 27 out of 100, grade F	11 blocked	Amazon CloudFront
2720	archlinux.org	Score 65 out of 100, grade C	all allowed	unidentified
2728	pypi.org	Score 67 out of 100, grade C	all allowed	Fastly
2729	centos.org	Score 59 out of 100, grade D	all allowed	unidentified
2745	opensource.org	Score 67 out of 100, grade C	all allowed	WordPress . Cloudflare
2758	wikisource.org	Score 67 out of 100, grade C	all allowed	unidentified
2764	trs-dns.org	Not scored	all allowed	unidentified
2773	ilo.orgrefused GPTBot	Score 52 out of 100, grade D	all allowed	Drupal . Cloudflare
2836	undp.org	Score 89 out of 100, grade B, partial assessment	all allowed	Akamai
2839	iop.orgrefused GPTBot	Score 89 out of 100, grade B, partial assessment	all allowed	Cloudflare
2870	phys.org	Score 91 out of 100, grade A	all allowed	unidentified
2932	heart.orgrefused GPTBot	Score 83 out of 100, grade B, partial assessment	all allowed	Cloudflare
2964	rsc.org	Score 67 out of 100, grade C	all allowed	Cloudflare
2983	iea.orgrefused GPTBot	Score 83 out of 100, grade B, partial assessment	all allowed	Cloudflare
2994	aarp.orgrefused GPTBot	Score 60 out of 100, grade C	all allowed	Adobe Experience Manager . Amazon CloudFront
2997	assetsconfigcdn.orgrefused GPTBot	Score 83 out of 100, grade B, partial assessment	all allowed	Google Cloud / GFE
3002	joomla.org	Score 67 out of 100, grade C	all allowed	Joomla . Cloudflare
3020	ohchr.orgrefused GPTBot	Score 60 out of 100, grade C	all allowed	Drupal . Cloudflare
3036	faqs.org	Score 53 out of 100, grade D	1 blocked	unidentified
3044	npmjs.orgrefused GPTBot	Score 83 out of 100, grade B, partial assessment	all allowed	Cloudflare
3045	cpan.org	Score 59 out of 100, grade D	all allowed	Fastly
3048	ntppool.org	Score 56 out of 100, grade D	all allowed	Fastly
3064	imagemagick.org	Score 70 out of 100, grade C	all allowed	Fastly
3074	dmoz.org	Not scored	all allowed	unidentified
3096	drpc.org	Score 95 out of 100, grade A	all allowed	Next.js . Cloudflare
3099	userway.org	Score 79 out of 100, grade B	all allowed	Gatsby . Amazon CloudFront
3154	wikibooks.org	Score 67 out of 100, grade C	all allowed	unidentified
3237	cancer.org	Score 71 out of 100, grade C	all allowed	Adobe Experience Manager . Fastly
3239	mediawiki.org	Score 71 out of 100, grade C	all allowed	unidentified
3243	geeksforgeeks.org	Score 71 out of 100, grade C	all allowed	WordPress . Amazon CloudFront
3262	videolan.org	Score 53 out of 100, grade D	all allowed	unidentified
3273	wireshark.org	Score 62 out of 100, grade C	all allowed	Astro . Cloudflare
3275	freedesktop.org	Not scored	all allowed	unidentified
3276	unep.org	Score 68 out of 100, grade C	all allowed	Drupal . Cloudflare
3287	greenpeace.org	Score 54 out of 100, grade D	all allowed	Cloudflare
3295	isc.org	Score 62 out of 100, grade C	all allowed	Fastly
3310	consumerreports.orgrefused GPTBot	Score 61 out of 100, grade C	2 blocked	Adobe Experience Manager . Amazon CloudFront
3318	openldap.org	Score 56 out of 100, grade D	all allowed	unidentified
3348	hrw.org	Score 66 out of 100, grade C	all allowed	Drupal . Fastly
3366	gnome.org	Score 59 out of 100, grade D	all allowed	Fastly
3384	hopkinsmedicine.orgrefused GPTBot	Score 83 out of 100, grade B, partial assessment	all allowed	Cloudflare
3413	nobelprize.orgrefused GPTBot	Score 65 out of 100, grade C	all allowed	WordPress . Cloudflare
3442	edx.org	Score 80 out of 100, grade B	all allowed	Next.js . Cloudflare
3453	collegeboard.org	Score 64 out of 100, grade C	all allowed	Drupal . Akamai
3493	mariadb.org	Score 80 out of 100, grade B	all allowed	WordPress
3497	code.org	Score 68 out of 100, grade C	all allowed	Next.js . Amazon CloudFront
3553	gimp.org	Score 62 out of 100, grade C	all allowed	Fastly
3607	lua.org	Score 45 out of 100, grade D	all allowed	unidentified
3612	ourworldindata.org	Score 64 out of 100, grade C	all allowed	Cloudflare
3644	man7.org	Score 52 out of 100, grade D	all allowed	unidentified
3654	amnesty.org	Score 78 out of 100, grade B	all allowed	WordPress . Cloudflare
3684	metmuseum.orgrefused GPTBot	Score 83 out of 100, grade B, partial assessment	all allowed	Vercel
3691	opensuse.orgrefused GPTBot	Score 49 out of 100, grade D	all allowed	unidentified
3701	worldcat.orgrefused GPTBot	Score 52 out of 100, grade D	3 blocked	Adobe Experience Manager . Cloudflare
3723	owasp.org	Score 62 out of 100, grade C	all allowed	Cloudflare
3828	zenodo.org	Score 59 out of 100, grade D	all allowed	unidentified
3848	familysearch.org	Score 65 out of 100, grade C	all allowed	Amazon CloudFront
3851	cfdata.org	Not scored	all allowed	unidentified
3869	codeberg.org	Score 33 out of 100, grade F, partial assessment	8 blocked	unidentified
3870	unhcr.orgrefused GPTBot	Score 100 out of 100, grade A, partial assessment	all allowed	Cloudflare
3880	readthedocs.orgrefused GPTBot	Score 52 out of 100, grade D	all allowed	Cloudflare
3887	eurekalert.org	Score 63 out of 100, grade C	all allowed	unidentified
3901	ispconfig.org	Score 64 out of 100, grade C	all allowed	WordPress . Cloudflare
3913	raspberrypi.org	Score 67 out of 100, grade C	all allowed	Contentful . Cloudflare
4034	js.org	Score 56 out of 100, grade D	all allowed	Cloudflare
4047	redcross.org	Score 56 out of 100, grade D	all allowed	Adobe Experience Manager . Akamai
4100	emojipedia.orgrefused GPTBot	Score 55 out of 100, grade D	4 blocked	Ghost . Cloudflare
4106	ahajournals.orgrefused GPTBot	Score 94 out of 100, grade A, partial assessment	1 blocked	Cloudflare
4122	ghost.org	Score 69 out of 100, grade C, partial assessment	all allowed	Netlify
4160	openweathermap.org	Score 71 out of 100, grade C	all allowed	Next.js
4161	eclipse.org	Score 59 out of 100, grade D	all allowed	unidentified
4182	britishcouncil.org	Not scored	all allowed	unidentified
4183	torproject.org	Score 59 out of 100, grade D	all allowed	unidentified
4196	tvtropes.orgrefused GPTBot	Score 83 out of 100, grade B, partial assessment	all allowed	Cloudflare
4198	us.org	Score 56 out of 100, grade D	all allowed	unidentified
4216	squid-cache.org	Score 59 out of 100, grade D	all allowed	unidentified
4222	blender.orgrefused GPTBot	Score 39 out of 100, grade F	5 blocked	WordPress . Cloudflare
4242	eu.org	Score 59 out of 100, grade D	all allowed	unidentified
4255	propublica.org	Score 83 out of 100, grade B	all allowed	WordPress . Cloudflare
4281	purl.org	Score 59 out of 100, grade D	all allowed	unidentified
4339	chromium.org	Score 53 out of 100, grade D	all allowed	Fastly
4383	nber.org	Score 62 out of 100, grade C	all allowed	Drupal . Fastly
4390	doxygen.org	Score 59 out of 100, grade D	all allowed	Cloudflare

Showing the 200 most-visited.

Using these figures

Free to reuse in research, journalism or a product under CC BY 4.0, with credit to Fidget Labs BV. Quote the measurement date so the claim stays checkable as the index moves.

CrawlIndex by Fidget Labs BV. "Agent readiness on .org." https://crawlindex.org (measured 2026-08-09). Licensed CC BY 4.0.