Top-level domain

.org

Name: CrawlIndex dataset
License: https://creativecommons.org/licenses/by/4.0/

216 measured sites. 23 of them, 10.6%, block at least one answer-surface AI crawler. Mean readiness score 65.4.

Compare against every other top-level domain

Sites on .org
Rank	Domain	Score	Answer-surface crawlers	Stack
30	wikipedia.org	Score 66 out of 100, grade C	all allowed	unidentified
48	wordpress.org	Score 89 out of 100, grade B	all allowed	WordPress
68	ntp.org	Score 55 out of 100, grade D	all allowed	unidentified
75	nginx.org	Score 62 out of 100, grade C	all allowed	unidentified
100	mozilla.org	Score 60 out of 100, grade C	all allowed	Fastly
131	apache.org	Score 60 out of 100, grade C	all allowed	Fastly
150	archive.org	Score 92 out of 100, grade A, partial assessment	all allowed	unidentified
183	base.org	Score 70 out of 100, grade C	all allowed	Next.js . Cloudflare
191	w3.org	Score 84 out of 100, grade B, partial assessment	all allowed	Cloudflare
196	lencr.org	Score 54 out of 100, grade D	all allowed	Netlify
198	creativecommons.org	Score 73 out of 100, grade C	all allowed	WordPress . Cloudflare
200	duckdns.org	Score 54 out of 100, grade D	all allowed	unidentified
213	doi.org	Score 54 out of 100, grade D	all allowed	Cloudflare
255	wikimedia.org	Score 54 out of 100, grade D	all allowed	unidentified
261	telegram.org	Score 52 out of 100, grade D	all allowed	unidentified
318	debian.org	Score 57 out of 100, grade D	all allowed	unidentified
347	gnu.orgrefused GPTBot	Score 54 out of 100, grade D	all allowed	unidentified
360	un.orgrefused GPTBot	Score 45 out of 100, grade D	all allowed	Drupal . Amazon CloudFront
373	cookiedatabase.orgrefused GPTBot	Score 78 out of 100, grade B, partial assessment	all allowed	Cloudflare
406	ipify.org	Score 58 out of 100, grade D	all allowed	HubSpot CMS . Amazon CloudFront
479	arxiv.org	Score 57 out of 100, grade D	all allowed	Fastly
493	ietf.orgrefused GPTBot	Score 53 out of 100, grade D	all allowed	Cloudflare
504	npr.org	Score 57 out of 100, grade D	all allowed	Akamai
521	ampproject.org	Score 57 out of 100, grade D	all allowed	Netlify
585	cookielaw.orgrefused GPTBot	Score 63 out of 100, grade C	all allowed	Adobe Experience Manager . Cloudflare
592	openstreetmap.orgrefused GPTBot	Score 48 out of 100, grade D	all allowed	Fastly
609	ieee.org	Score 78 out of 100, grade B, partial assessment	all allowed	Amazon CloudFront
633	unesco.org	Score 68 out of 100, grade C	4 blocked	Drupal
636	dyndns.org	Score 52 out of 100, grade D	all allowed	unidentified
683	cambridge.orgrefused GPTBot	Score 86 out of 100, grade B, partial assessment	1 blocked	Cloudflare
715	iso.orgrefused GPTBot	Score 78 out of 100, grade B, partial assessment	all allowed	Cloudflare
752	python.org	Score 65 out of 100, grade C	all allowed	Fastly
769	featureassets.orgrefused GPTBot	Score 78 out of 100, grade B, partial assessment	all allowed	Google Cloud / GFE
791	craigslist.org	Score 60 out of 100, grade C	all allowed	unidentified
794	allaboutcookies.org	Score 78 out of 100, grade B, partial assessment	all allowed	Cloudflare
813	networkadvertising.org	Score 70 out of 100, grade C	all allowed	WordPress . Cloudflare
880	hbr.org	Score 54 out of 100, grade D	2 blocked	Next.js . Amazon CloudFront
882	frontiersin.org	Score 83 out of 100, grade B	all allowed	Nuxt . Azure Front Door
901	mayoclinic.org	Score 78 out of 100, grade B, partial assessment	all allowed	Akamai
908	worldbank.org	Score 82 out of 100, grade B	all allowed	Adobe Experience Manager . Cloudflare
957	thenai.org	Score 70 out of 100, grade C	all allowed	WordPress . Cloudflare
976	change.org	Score 74 out of 100, grade C	all allowed	Amazon CloudFront
1008	weforum.org	Score 78 out of 100, grade B, partial assessment	all allowed	Akamai
1025	spamhaus.org	Score 60 out of 100, grade C	all allowed	Nuxt . Cloudflare
1027	ecosia.orgrefused GPTBot	Score 84 out of 100, grade B, partial assessment	all allowed	Cloudflare
1046	archiveofourown.orgrefused GPTBot	Score 47 out of 100, grade D	2 blocked	Cloudflare
1056	pbs.org	Score 64 out of 100, grade C	2 blocked	Next.js . Amazon CloudFront
1070	adblockplus.org	Score 54 out of 100, grade D	all allowed	Fastly
1097	oecd.orgrefused GPTBot	Score 92 out of 100, grade A, partial assessment	all allowed	Cloudflare
1133	hltv.orgrefused GPTBot	Score 78 out of 100, grade B, partial assessment	all allowed	Cloudflare
1145	apa.orgrefused GPTBot	Score 64 out of 100, grade C	all allowed	Imperva
1151	letsencrypt.org	Score 57 out of 100, grade D	all allowed	Netlify
1159	rfc-editor.org	Score 64 out of 100, grade C	all allowed	Nuxt . Cloudflare
1178	plos.org	Score 80 out of 100, grade B	all allowed	HubSpot CMS . Cloudflare
1184	jw.org	Score 61 out of 100, grade C	all allowed	Amazon CloudFront
1223	coursera.org	Score 80 out of 100, grade B	1 blocked	Contentful . Amazon CloudFront
1227	lichess.org	Score 60 out of 100, grade C	all allowed	unidentified
1235	wiktionary.org	Score 66 out of 100, grade C	all allowed	unidentified
1275	kernel.org	Score 54 out of 100, grade D	all allowed	unidentified
1284	bbb.org	Score 78 out of 100, grade B, partial assessment	all allowed	Cloudflare
1287	themoviedb.org	Score 24 out of 100, grade F	11 blocked	Amazon CloudFront
1297	postgresql.org	Score 67 out of 100, grade C	all allowed	Varnish
1334	jstor.org	Score 74 out of 100, grade C, partial assessment	3 blocked	Fastly
1339	tgju.org	Score 74 out of 100, grade C	all allowed	Cloudflare
1340	aternos.orgrefused GPTBot	Score 84 out of 100, grade B, partial assessment	all allowed	Cloudflare
1370	pornhub.org	Score 69 out of 100, grade C	all allowed	unidentified
1375	e-hentai.org	Score 62 out of 100, grade C	all allowed	Cloudflare
1432	nosv.org	Score 58 out of 100, grade D	all allowed	unidentified
1437	iana.org	Score 58 out of 100, grade D	all allowed	Cloudflare
1468	fedoraproject.orgrefused GPTBot	Score 60 out of 100, grade C	all allowed	Nuxt
1515	exhentai.org	Score 6 out of 100, grade F, partial assessment	11 blocked	unidentified
1548	tjk.org	Score 6 out of 100, grade F, partial assessment	11 blocked	unidentified
1578	syosetu.orgrefused GPTBot	Score 78 out of 100, grade B, partial assessment	all allowed	Cloudflare
1608	anybunny.org	Score 55 out of 100, grade D	all allowed	Cloudflare
1638	acm.orgrefused GPTBot	Score 92 out of 100, grade A, partial assessment	all allowed	Cloudflare
1700	eff.org	Score 55 out of 100, grade D	all allowed	Drupal . Fastly
1734	xxxpostpic.org	Score 77 out of 100, grade B	all allowed	WordPress . Cloudflare
1746	newtoki1.org	Not scored	all allowed	unidentified
1797	fao.org	Score 70 out of 100, grade C	all allowed	Cloudflare
1820	acs.org	Score 67 out of 100, grade C	all allowed	Adobe Experience Manager . Fastly
1842	moodle.orgrefused GPTBot	Score 88 out of 100, grade B, partial assessment	all allowed	Cloudflare
1852	pewresearch.org	Score 79 out of 100, grade B	all allowed	WordPress
1889	clevelandclinic.org	Score 73 out of 100, grade C	all allowed	Next.js . Cloudflare
1903	sciencemag.orgrefused GPTBot	Score 68 out of 100, grade C, partial assessment	4 blocked	Cloudflare
1907	orcid.org	Score 48 out of 100, grade D	all allowed	Cloudflare
1943	pnas.orgrefused GPTBot	Score 86 out of 100, grade B, partial assessment	1 blocked	Cloudflare
1956	mitre.org	Score 78 out of 100, grade B, partial assessment	all allowed	unidentified
1980	freebsd.org	Score 60 out of 100, grade C	all allowed	unidentified
1996	science.orgrefused GPTBot	Score 68 out of 100, grade C, partial assessment	4 blocked	Cloudflare
2043	bitbucket.orgrefused GPTBot	Score 71 out of 100, grade C	all allowed	Amazon CloudFront
2080	aboutcookies.org	Score 71 out of 100, grade C	all allowed	Sitecore . Cloudflare
2089	unicef.org	Score 60 out of 100, grade C	all allowed	Drupal . Cloudflare
2112	openssl.org	Score 6 out of 100, grade F, partial assessment	11 blocked	unidentified
2120	nodejs.org	Score 73 out of 100, grade C	all allowed	Next.js . Cloudflare
2150	semanticscholar.orgrefused GPTBot	Score 54 out of 100, grade D	all allowed	HubSpot CMS . Amazon CloudFront
2220	khanacademy.org	Score 78 out of 100, grade B, partial assessment	all allowed	Varnish
2245	icann.orgrefused GPTBot	Score 84 out of 100, grade B, partial assessment	all allowed	Cloudflare
2250	kde.org	Score 64 out of 100, grade C	all allowed	unidentified
2251	signal.org	Score 57 out of 100, grade D	all allowed	Cloudflare
2271	afraid.org	Score 60 out of 100, grade C	all allowed	unidentified
2329	altervista.org	Score 64 out of 100, grade C	all allowed	Amazon CloudFront
2371	opensource.orgrefused GPTBot	Score 57 out of 100, grade D	all allowed	WordPress . Cloudflare
2392	nejm.orgrefused GPTBot	Score 68 out of 100, grade C, partial assessment	4 blocked	Cloudflare
2395	pypi.org	Score 64 out of 100, grade C	all allowed	Fastly
2396	imf.org	Score 92 out of 100, grade A, partial assessment	all allowed	Akamai
2458	apachefriends.org	Score 61 out of 100, grade C	all allowed	Fastly
2534	collegeboard.org	Score 62 out of 100, grade C	all allowed	Drupal . Akamai
2565	slashdot.org	Score 73 out of 100, grade C	all allowed	WordPress . Cloudflare
2590	cve.org	Score 92 out of 100, grade A, partial assessment	all allowed	Amazon CloudFront
2600	perl.org	Score 55 out of 100, grade D	all allowed	Fastly
2604	drupal.org	Score 84 out of 100, grade B, partial assessment	all allowed	Fastly
2605	matomo.org	Score 80 out of 100, grade B	all allowed	WordPress . Amazon CloudFront
2611	tizen.orgrefused GPTBot	Score 78 out of 100, grade B, partial assessment	all allowed	Amazon CloudFront
2635	tmdb.org	Score 24 out of 100, grade F	11 blocked	Amazon CloudFront
2670	gutenberg.org	Score 58 out of 100, grade D	all allowed	unidentified
2683	archlinux.org	Score 62 out of 100, grade C	all allowed	unidentified
2716	npmjs.orgrefused GPTBot	Score 78 out of 100, grade B, partial assessment	all allowed	Cloudflare
2727	centos.org	Score 57 out of 100, grade D	all allowed	unidentified
2798	wikisource.org	Score 66 out of 100, grade C	all allowed	unidentified
2834	ilo.orgrefused GPTBot	Score 59 out of 100, grade D	all allowed	Drupal . Cloudflare
2912	aarp.orgrefused GPTBot	Score 60 out of 100, grade C	all allowed	Adobe Experience Manager . Amazon CloudFront
2917	iop.orgrefused GPTBot	Score 84 out of 100, grade B, partial assessment	all allowed	Cloudflare
2926	undp.org	Score 84 out of 100, grade B, partial assessment	all allowed	unidentified
2990	phys.org	Score 83 out of 100, grade B	all allowed	unidentified
2991	iea.orgrefused GPTBot	Score 78 out of 100, grade B, partial assessment	all allowed	Cloudflare
3004	assetsconfigcdn.orgrefused GPTBot	Score 78 out of 100, grade B, partial assessment	all allowed	Google Cloud / GFE
3031	heart.orgrefused GPTBot	Score 78 out of 100, grade B, partial assessment	all allowed	Cloudflare
3061	joomla.org	Score 66 out of 100, grade C	all allowed	Joomla . Cloudflare
3083	cpan.org	Score 60 out of 100, grade C	all allowed	Fastly
3085	rsc.org	Score 64 out of 100, grade C	all allowed	Cloudflare
3088	userway.org	Score 74 out of 100, grade C	all allowed	Gatsby . Amazon CloudFront
3095	ntppool.org	Score 55 out of 100, grade D	all allowed	Fastly
3102	faqs.orgrefused GPTBot	Score 78 out of 100, grade B, partial assessment	1 blocked	unidentified
3135	ohchr.orgrefused GPTBot	Score 62 out of 100, grade C	all allowed	Drupal . Cloudflare
3156	imagemagick.org	Score 66 out of 100, grade C	all allowed	Fastly
3177	openldap.org	Score 57 out of 100, grade D	all allowed	unidentified
3243	mediawiki.org	Score 74 out of 100, grade C	all allowed	unidentified
3288	geeksforgeeks.org	Score 67 out of 100, grade C	all allowed	WordPress . Amazon CloudFront
3325	isc.org	Score 62 out of 100, grade C	all allowed	Fastly
3329	unep.org	Score 65 out of 100, grade C	all allowed	Drupal . Cloudflare
3349	wikibooks.org	Score 66 out of 100, grade C	all allowed	unidentified
3354	videolan.org	Score 54 out of 100, grade D	all allowed	unidentified
3389	worldcat.orgrefused GPTBot	Score 48 out of 100, grade D	3 blocked	Adobe Experience Manager . Cloudflare
3403	cancer.org	Score 67 out of 100, grade C	all allowed	Adobe Experience Manager . Fastly
3417	wireshark.org	Score 60 out of 100, grade C	all allowed	Astro . Cloudflare
3442	gnome.org	Score 57 out of 100, grade D	all allowed	Fastly
3454	familysearch.org	Score 61 out of 100, grade C	all allowed	Amazon CloudFront
3470	greenpeace.org	Score 56 out of 100, grade D	all allowed	Cloudflare
3472	consumerreports.orgrefused GPTBot	Score 57 out of 100, grade D	2 blocked	Adobe Experience Manager . Amazon CloudFront
3555	owasp.org	Score 56 out of 100, grade D	all allowed	Next.js . Vercel
3572	hrw.org	Score 63 out of 100, grade C	all allowed	Drupal . Fastly
3579	drpc.org	Score 86 out of 100, grade B	all allowed	Next.js . Cloudflare
3596	nobelprize.orgrefused GPTBot	Score 63 out of 100, grade C	all allowed	WordPress . Cloudflare
3601	edx.org	Score 74 out of 100, grade C	all allowed	Next.js . Cloudflare
3610	hopkinsmedicine.orgrefused GPTBot	Score 78 out of 100, grade B, partial assessment	all allowed	Cloudflare
3628	zenodo.org	Score 57 out of 100, grade D	all allowed	unidentified
3641	man7.org	Score 52 out of 100, grade D	all allowed	unidentified
3674	mariadb.org	Score 77 out of 100, grade B	all allowed	WordPress
3680	gimp.org	Score 60 out of 100, grade C	all allowed	Fastly
3687	lua.org	Score 45 out of 100, grade D	all allowed	unidentified
3749	js.org	Score 54 out of 100, grade D	all allowed	Cloudflare
3771	codeberg.org	Score 31 out of 100, grade F, partial assessment	8 blocked	unidentified
3799	amnesty.org	Score 76 out of 100, grade B	all allowed	WordPress . Cloudflare
3819	metmuseum.orgrefused GPTBot	Score 78 out of 100, grade B, partial assessment	all allowed	Vercel
3831	opensuse.orgrefused GPTBot	Score 48 out of 100, grade D	all allowed	unidentified
3855	ispconfig.org	Score 64 out of 100, grade C	all allowed	WordPress . Cloudflare
3893	ourworldindata.org	Score 61 out of 100, grade C	all allowed	Cloudflare
4027	readthedocs.orgrefused GPTBot	Score 53 out of 100, grade D	all allowed	Cloudflare
4048	wikidata.org	Score 66 out of 100, grade C	all allowed	unidentified
4054	unhcr.orgrefused GPTBot	Score 92 out of 100, grade A, partial assessment	all allowed	Cloudflare
4066	code.org	Score 74 out of 100, grade C	all allowed	Next.js . Amazon CloudFront
4071	redcross.org	Score 54 out of 100, grade D	all allowed	Adobe Experience Manager . Akamai
4083	torproject.org	Score 57 out of 100, grade D	all allowed	unidentified
4199	ghost.org	Score 67 out of 100, grade C	all allowed	Netlify
4203	openoffice.org	Score 58 out of 100, grade D	all allowed	unidentified
4238	ahajournals.orgrefused GPTBot	Score 86 out of 100, grade B, partial assessment	1 blocked	Cloudflare
4242	raspberrypi.org	Score 69 out of 100, grade C	all allowed	Contentful . Cloudflare
4265	emojipedia.orgrefused GPTBot	Score 53 out of 100, grade D	4 blocked	Ghost . Cloudflare
4272	squid-cache.org	Score 58 out of 100, grade D	all allowed	unidentified
4286	propublica.org	Score 80 out of 100, grade B	all allowed	WordPress . Cloudflare
4340	purl.org	Score 58 out of 100, grade D	all allowed	unidentified
4353	chromium.org	Score 92 out of 100, grade A, partial assessment	all allowed	Fastly
4357	tvtropes.orgrefused GPTBot	Score 78 out of 100, grade B, partial assessment	all allowed	Cloudflare
4396	blender.orgrefused GPTBot	Score 59 out of 100, grade D	all allowed	WordPress . Cloudflare
4423	nber.org	Score 63 out of 100, grade C	all allowed	Drupal . Fastly
4447	doxygen.org	Score 58 out of 100, grade D	all allowed	Cloudflare
4490	nessus.org	Score 59 out of 100, grade D	all allowed	Drupal . Cloudflare
4503	eclipse.org	Score 57 out of 100, grade D	all allowed	unidentified
4519	libpng.org	Score 58 out of 100, grade D	all allowed	unidentified
4573	virtualbox.org	Score 55 out of 100, grade D	all allowed	Akamai
4589	filezilla-project.orgrefused GPTBot	Score 84 out of 100, grade B, partial assessment	all allowed	unidentified
4592	memcached.org	Score 55 out of 100, grade D	all allowed	unidentified
4615	packagist.org	Score 62 out of 100, grade C	all allowed	Bunny
4630	pcre.org	Score 84 out of 100, grade B, partial assessment	all allowed	unidentified
4685	oxfordjournals.orgrefused GPTBot	Score 92 out of 100, grade A, partial assessment	all allowed	Cloudflare
4699	biorxiv.org	Score 45 out of 100, grade D	4 blocked	Drupal . Cloudflare
4715	gzip.org	Score 55 out of 100, grade D	all allowed	unidentified
4721	rfa.org	Score 68 out of 100, grade C	all allowed	Akamai
4751	sqlite.orgrefused GPTBot	Score 48 out of 100, grade D	all allowed	unidentified
4766	freetype.org	Score 58 out of 100, grade D	all allowed	Cloudflare

Showing the 200 most-visited.

Using these figures

Free to reuse in research, journalism or a product under CC BY 4.0, with credit to Fidget Labs BV. Quote the measurement date so the claim stays checkable as the index moves.

CrawlIndex by Fidget Labs BV. "Agent readiness on .org." https://crawlindex.org (measured 2026-09-22). Licensed CC BY 4.0.