Top-level domain

.net

Name: CrawlIndex dataset
License: https://creativecommons.org/licenses/by/4.0/

181 measured sites. 32 of them, 17.7%, block at least one answer-surface AI crawler. Mean readiness score 61.6.

Compare against every other top-level domain

Sites on .net
Rank	Domain	Score	Answer-surface crawlers	Stack
36	fastly.net	Score 80 out of 100, grade B	all allowed	Fastly
37	doubleclick.net	Score 67 out of 100, grade C	all allowed	unidentified
44	gandi.net	Score 77 out of 100, grade B	all allowed	Fastly
57	cloudflare.netrefused GPTBot	Score 78 out of 100, grade B, partial assessment	all allowed	Cloudflare
71	myfritz.net	Score 14 out of 100, grade F, partial assessment	11 blocked	unidentified
84	yandex.net	Not scored	all allowed	unidentified
108	b-cdn.net	Score 59 out of 100, grade D	all allowed	Bunny
180	telekom.net	Score 59 out of 100, grade D	all allowed	Adobe Experience Manager . Fastly
225	researchgate.netrefused GPTBot	Score 84 out of 100, grade B, partial assessment	all allowed	Cloudflare
240	cpanel.net	Score 77 out of 100, grade B	all allowed	WordPress . Cloudflare
242	statsigapi.netrefused GPTBot	Score 78 out of 100, grade B, partial assessment	all allowed	Google Cloud / GFE
248	sourceforge.net	Score 92 out of 100, grade A, partial assessment	all allowed	Cloudflare
250	comcast.net	Score 78 out of 100, grade B, partial assessment	all allowed	unidentified
277	launchpad.netrefused GPTBot	Score 20 out of 100, grade F	9 blocked	unidentified
328	php.net	Score 60 out of 100, grade C	all allowed	Bunny
337	trueconf.net	Score 71 out of 100, grade C	all allowed	WordPress
370	live-video.net	Score 52 out of 100, grade D	all allowed	Amazon CloudFront
411	nflxvideo.net	Score 59 out of 100, grade D, partial assessment	3 blocked	unidentified
454	cloudns.net	Score 67 out of 100, grade C	all allowed	unidentified
469	speedtest.netrefused GPTBot	Score 84 out of 100, grade B, partial assessment	all allowed	Cloudflare
489	nease.net	Score 78 out of 100, grade B, partial assessment	all allowed	unidentified
507	atlassian.net	Score 78 out of 100, grade B	all allowed	Amazon CloudFront
514	behance.net	Score 60 out of 100, grade C, partial assessment	5 blocked	Fastly
530	slideshare.net	Score 78 out of 100, grade B, partial assessment	all allowed	Fastly
550	netflix.net	Score 59 out of 100, grade D, partial assessment	3 blocked	unidentified
555	bitdefender.net	Score 57 out of 100, grade D	all allowed	Adobe Experience Manager . Cloudflare
606	xiaomi.net	Score 64 out of 100, grade C	all allowed	Akamai
644	ovh.net	Score 45 out of 100, grade D, partial assessment	all allowed	unidentified
668	hstgr.net	Score 6 out of 100, grade F, partial assessment	11 blocked	unidentified
785	singular.net	Score 89 out of 100, grade B	all allowed	HubSpot CMS . Cloudflare
817	adform.net	Score 6 out of 100, grade F, partial assessment	11 blocked	unidentified
826	a-mo.net	Score 78 out of 100, grade B, partial assessment	all allowed	Amazon CloudFront
843	daum.net	Score 6 out of 100, grade F, partial assessment	11 blocked	unidentified
861	att.netrefused GPTBot	Score 50 out of 100, grade D	all allowed	Next.js
909	fwmrm.net	Score 70 out of 100, grade C	all allowed	HubSpot CMS . Cloudflare
916	internetwarriors.net	Score 52 out of 100, grade D	all allowed	unidentified
967	sentinelone.net	Not scored	all allowed	unidentified
985	ddns.net	Score 54 out of 100, grade D	all allowed	unidentified
1000	pixiv.netrefused GPTBot	Score 52 out of 100, grade D, partial assessment	5 blocked	Cloudflare
1034	themeforest.net	Score 70 out of 100, grade C	all allowed	Cloudflare
1131	nsone.net	Score 78 out of 100, grade B, partial assessment	all allowed	Cloudflare
1160	he.net	Score 59 out of 100, grade D	all allowed	unidentified
1162	yastatic.net	Not scored	all allowed	unidentified
1189	vnexpress.net	Score 70 out of 100, grade C	all allowed	unidentified
1240	savefrom.net	Score 77 out of 100, grade B	all allowed	Cloudflare
1241	blackberry.net	Score 66 out of 100, grade C	all allowed	Storyblok . Amazon CloudFront
1254	furaffinity.netrefused GPTBot	Score 6 out of 100, grade F, partial assessment	11 blocked	Cloudflare
1337	calculator.net	Score 62 out of 100, grade C	all allowed	unidentified
1343	charter.netrefused GPTBot	Score 45 out of 100, grade D	all allowed	Amazon CloudFront
1345	hollywoodbets.netrefused GPTBot	Score 78 out of 100, grade B, partial assessment	all allowed	Cloudflare
1348	jalan.net	Score 62 out of 100, grade C	all allowed	unidentified
1352	e621.net	Score 51 out of 100, grade D	4 blocked	Cloudflare
1363	inleed.netrefused GPTBot	Score 53 out of 100, grade D	all allowed	unidentified
1366	fdown.netrefused GPTBot	Score 92 out of 100, grade A, partial assessment	all allowed	Cloudflare
1406	zedge.net	Score 61 out of 100, grade C	1 blocked	Next.js . Cloudflare
1417	ficbook.netrefused GPTBot	Score 92 out of 100, grade A, partial assessment	all allowed	Cloudflare
1419	carsensor.net	Score 69 out of 100, grade C	all allowed	unidentified
1481	eenadu.net	Score 42 out of 100, grade D	11 blocked	Amazon CloudFront
1485	dogdrip.netrefused GPTBot	Score 48 out of 100, grade D	all allowed	Cloudflare
1493	iporntv.net	Score 55 out of 100, grade D	all allowed	Cloudflare
1518	programme-tv.net	Score 75 out of 100, grade B	all allowed	unidentified
1530	tokyomotion.net	Score 55 out of 100, grade D	all allowed	unidentified
1531	cityheaven.net	Score 78 out of 100, grade B, partial assessment	all allowed	unidentified
1557	theqoo.netrefused GPTBot	Score 78 out of 100, grade B, partial assessment	all allowed	Cloudflare
1562	therapservices.net	Score 90 out of 100, grade A	all allowed	WordPress . Cloudflare
1566	czbooks.netrefused GPTBot	Score 78 out of 100, grade B, partial assessment	all allowed	Cloudflare
1589	pussyboy.net	Score 55 out of 100, grade D	all allowed	Cloudflare
1621	girlschannel.net	Score 62 out of 100, grade C	all allowed	unidentified
1636	taktube.net	Score 74 out of 100, grade C	all allowed	WordPress
1641	kir2kos.net	Score 70 out of 100, grade C	all allowed	WordPress . Cloudflare
1664	xpaja.net	Score 67 out of 100, grade C	all allowed	unidentified
1714	kinnser.net	Not scored	all allowed	unidentified
1757	sexmupxinh.netrefused GPTBot	Score 92 out of 100, grade A, partial assessment	all allowed	Cloudflare
1782	threads.net	Score 14 out of 100, grade F, partial assessment	11 blocked	unidentified
1928	appier.net	Score 64 out of 100, grade C	all allowed	HubSpot CMS . Cloudflare
1976	secureserver.net	Score 6 out of 100, grade F, partial assessment	11 blocked	unidentified
1993	iponweb.net	Score 78 out of 100, grade B, partial assessment	all allowed	unidentified
1997	savvis.net	Score 60 out of 100, grade C	all allowed	Adobe Experience Manager . Fastly
2063	fortinet.net	Score 67 out of 100, grade C	all allowed	Adobe Experience Manager . Amazon CloudFront
2067	hubspot.net	Score 82 out of 100, grade B	all allowed	HubSpot CMS . Cloudflare
2085	av380.net	Not scored	all allowed	unidentified
2091	virtualearth.net	Score 48 out of 100, grade D	all allowed	unidentified
2121	earthlink.netrefused GPTBot	Score 78 out of 100, grade B, partial assessment	all allowed	Cloudflare
2171	chartbeat.net	Score 80 out of 100, grade B	all allowed	WordPress . Fastly
2187	im-apps.net	Score 49 out of 100, grade D	all allowed	Google Cloud / GFE
2229	ad-delivery.net	Score 49 out of 100, grade D	all allowed	Cloudflare
2240	mintegral.net	Score 55 out of 100, grade D	all allowed	Nuxt . Amazon CloudFront
2285	smi2.net	Score 58 out of 100, grade D	all allowed	Next.js
2293	bunny.net	Score 66 out of 100, grade C	all allowed	Bunny
2330	wordwall.net	Score 59 out of 100, grade D	all allowed	Azure Front Door
2347	footmercato.net	Score 80 out of 100, grade B	all allowed	Varnish
2435	artfight.netrefused GPTBot	Score 38 out of 100, grade F, partial assessment	7 blocked	Cloudflare
2439	avto.netrefused GPTBot	Score 78 out of 100, grade B, partial assessment	all allowed	Cloudflare
2443	gmx.net	Score 57 out of 100, grade D	all allowed	unidentified
2445	escortbabylon.net	Score 59 out of 100, grade D	all allowed	Cloudflare
2459	desiviral.net	Score 76 out of 100, grade B	all allowed	unidentified
2462	pubnative.net	Score 89 out of 100, grade B	all allowed	WordPress . Cloudflare
2468	churnzero.netrefused GPTBot	Score 73 out of 100, grade C	all allowed	HubSpot CMS
2470	lectormangass.netrefused GPTBot	Score 58 out of 100, grade D	4 blocked	Cloudflare
2572	csdn.net	Score 68 out of 100, grade C	all allowed	unidentified
2577	optimove.net	Score 52 out of 100, grade D	all allowed	Google Cloud / GFE
2581	expressapisv2.net	Score 6 out of 100, grade F, partial assessment	11 blocked	unidentified
2631	liquipedia.netrefused GPTBot	Score 66 out of 100, grade C, partial assessment	4 blocked	unidentified
2639	sensic.net	Score 49 out of 100, grade D	all allowed	Amazon CloudFront
2818	3hentai.net	Score 55 out of 100, grade D, partial assessment	1 blocked	unidentified
2864	minecraft.net	Score 67 out of 100, grade C	all allowed	Adobe Experience Manager . Akamai
2902	faz.netrefused GPTBot	Score 26 out of 100, grade F	10 blocked	Nuxt
2905	trafficjunky.net	Score 68 out of 100, grade C	all allowed	unidentified
2936	desikahani2.net	Score 82 out of 100, grade B	all allowed	WordPress . Cloudflare
2963	tamilsexvids.net	Score 73 out of 100, grade C	all allowed	WordPress . Cloudflare
3128	sc-static.netrefused GPTBot	Score 78 out of 100, grade B, partial assessment	all allowed	Amazon CloudFront
3198	yahoo.netrefused GPTBot	Score 47 out of 100, grade D	5 blocked	Next.js
3205	servidoresdns.net	Score 71 out of 100, grade C	all allowed	unidentified
3239	myanimelist.net	Score 46 out of 100, grade D	5 blocked	unidentified
3281	su89-cdn.net	Score 74 out of 100, grade C	all allowed	WordPress
3283	lausd.net	Not scored	all allowed	unidentified
3295	uk.net	Score 54 out of 100, grade D	all allowed	unidentified
3302	imgix.net	Score 58 out of 100, grade D	all allowed	HubSpot CMS . Cloudflare
3309	hinet.net	Not scored	all allowed	unidentified
3345	gaijin.net	Score 55 out of 100, grade D	all allowed	unidentified
3357	infojobs.net	Score 30 out of 100, grade F	11 blocked	Amazon CloudFront
3366	shawcable.net	Score 62 out of 100, grade C	all allowed	Contentful . Akamai
3394	signalr.net	Score 55 out of 100, grade D	all allowed	Azure Front Door
3401	wargaming.net	Score 62 out of 100, grade C	all allowed	unidentified
3438	ripe.net	Score 67 out of 100, grade C	all allowed	Akamai
3491	moviesdatamil.netrefused GPTBot	Score 52 out of 100, grade D	all allowed	Cloudflare
3518	c6gj-static.net	Score 74 out of 100, grade C	all allowed	WordPress
3535	serverdata.net	Score 55 out of 100, grade D	all allowed	unidentified
3559	reverso.netrefused GPTBot	Score 78 out of 100, grade B, partial assessment	all allowed	Cloudflare
3607	rs6.net	Score 6 out of 100, grade F, partial assessment	11 blocked	unidentified
3640	emarsys.netrefused GPTBot	Score 78 out of 100, grade B, partial assessment	all allowed	Cloudflare
3673	trouble-free.netrefused GPTBot	Score 78 out of 100, grade B, partial assessment	all allowed	Cloudflare
3689	grafana.net	Score 80 out of 100, grade B	all allowed	Storyblok . Vercel
3726	17track.net	Score 82 out of 100, grade B	all allowed	Next.js . Cloudflare
3733	handle.net	Score 55 out of 100, grade D	all allowed	Cloudflare
3739	consentmanager.net	Score 78 out of 100, grade B	all allowed	WordPress . Bunny
3746	aljazeera.net	Score 65 out of 100, grade C	4 blocked	WordPress
3758	4wps.netrefused GPTBot	Score 78 out of 100, grade B, partial assessment	all allowed	Amazon CloudFront
3776	entrust.net	Score 6 out of 100, grade F, partial assessment	11 blocked	unidentified
3786	atl-paas.net	Score 49 out of 100, grade D	all allowed	Amazon CloudFront
3906	arin.net	Score 67 out of 100, grade C	all allowed	unidentified
3910	apnic.net	Not scored	all allowed	unidentified
3958	jp.net	Score 52 out of 100, grade D	all allowed	unidentified
4042	quad9.net	Score 64 out of 100, grade C	all allowed	unidentified
4045	onenote.net	Score 48 out of 100, grade D	all allowed	unidentified
4078	uuidksinc.netrefused GPTBot	Score 78 out of 100, grade B, partial assessment	all allowed	unidentified
4119	proxad.net	Score 62 out of 100, grade C	all allowed	Next.js
4154	mailgun.net	Score 78 out of 100, grade B	all allowed	WordPress . Cloudflare
4161	a1.netrefused GPTBot	Score 78 out of 100, grade B, partial assessment	all allowed	unidentified
4165	126.net	Score 52 out of 100, grade D	all allowed	unidentified
4184	cox.net	Score 78 out of 100, grade B, partial assessment	all allowed	Imperva
4192	ucoz.net	Score 63 out of 100, grade C	all allowed	unidentified
4244	rtmark.net	Score 52 out of 100, grade D	all allowed	unidentified
4276	zlib.net	Score 58 out of 100, grade D	all allowed	unidentified
4297	localto.net	Score 52 out of 100, grade D	all allowed	unidentified
4391	skyscanner.netrefused GPTBot	Score 45 out of 100, grade D	1 blocked	Amazon CloudFront
4397	thunderbird.netrefused GPTBot	Score 78 out of 100, grade B, partial assessment	all allowed	Cloudflare
4406	cookieadmin.net	Score 76 out of 100, grade B	all allowed	Cloudflare
4435	cnki.net	Score 6 out of 100, grade F, partial assessment	11 blocked	unidentified
4471	alarabiya.netrefused GPTBot	Score 78 out of 100, grade B, partial assessment	all allowed	Cloudflare
4484	apkpure.net	Score 92 out of 100, grade A, partial assessment	all allowed	Cloudflare
4504	paycomonline.net	Score 6 out of 100, grade F, partial assessment	11 blocked	unidentified
4517	nhentai.netrefused GPTBot	Score 92 out of 100, grade A, partial assessment	all allowed	Cloudflare
4522	syncthing.net	Score 52 out of 100, grade D	all allowed	Fastly
4555	gb.net	Score 54 out of 100, grade D	all allowed	unidentified
4568	adspower.net	Score 91 out of 100, grade A	all allowed	Amazon CloudFront
4587	myhostadmin.net	Score 55 out of 100, grade D	all allowed	unidentified
4594	intermedia.net	Score 80 out of 100, grade B	all allowed	WordPress . Cloudflare
4632	cableone.netrefused GPTBot	Score 78 out of 100, grade B, partial assessment	all allowed	Vercel
4637	meituan.net	Score 45 out of 100, grade D	all allowed	unidentified
4663	aiv-cdn.net	Not scored	all allowed	unidentified
4711	grammarly.net	Score 64 out of 100, grade C	all allowed	Contentful . Amazon CloudFront
4717	srv53.net	Score 57 out of 100, grade D	all allowed	unidentified
4728	ezoic.net	Score 57 out of 100, grade D	all allowed	Next.js . Vercel
4739	ampproject.net	Score 57 out of 100, grade D	all allowed	Netlify
4836	t0x.net	Score 84 out of 100, grade B, partial assessment	all allowed	unidentified
4842	hardened-php.net	Score 92 out of 100, grade A, partial assessment	all allowed	unidentified
4851	verisign.net	Score 61 out of 100, grade C	all allowed	Amazon CloudFront
4897	yandexadexchange.net	Score 6 out of 100, grade F, partial assessment	11 blocked	unidentified
4968	finanzen.net	Score 74 out of 100, grade C	all allowed	WordPress
4981	spamcop.net	Score 55 out of 100, grade D	all allowed	unidentified
5002	tds.net	Score 31 out of 100, grade F	5 blocked	Varnish
5003	baishan-cloud.net	Score 55 out of 100, grade D	all allowed	unidentified
5013	html5up.net	Score 60 out of 100, grade C	all allowed	Cloudflare
5026	spectrum.netrefused GPTBot	Score 45 out of 100, grade D	all allowed	Amazon CloudFront
5044	openxcdn.net	Score 78 out of 100, grade B, partial assessment	all allowed	unidentified
5047	clara.net	Score 83 out of 100, grade B	all allowed	Astro
5060	mcafee.net	Score 78 out of 100, grade B, partial assessment	all allowed	Akamai
5067	phpmyadmin.net	Score 66 out of 100, grade C	all allowed	unidentified
5109	alikunlun.net	Not scored	all allowed	unidentified
5120	widgetable.net	Score 84 out of 100, grade B, partial assessment	all allowed	Amazon CloudFront

Using these figures

Free to reuse in research, journalism or a product under CC BY 4.0, with credit to Fidget Labs BV. Quote the measurement date so the claim stays checkable as the index moves.

CrawlIndex by Fidget Labs BV. "Agent readiness on .net." https://crawlindex.org (measured 2026-09-22). Licensed CC BY 4.0.