Top-level domain

.edu

Name: CrawlIndex dataset
License: https://creativecommons.org/licenses/by/4.0/

78 measured sites. 3 of them, 3.8%, block at least one answer-surface AI crawler. Mean readiness score 65.8.

Compare against every other top-level domain

Sites on .edu
Rank	Domain	Score	Answer-surface crawlers	Stack
229	mit.edu	Score 60 out of 100, grade C	all allowed	Akamai
319	harvard.edu	Score 72 out of 100, grade C	all allowed	WordPress
381	stanford.edu	Score 67 out of 100, grade C	all allowed	Storyblok . Netlify
577	cornell.edurefused GPTBot	Score 58 out of 100, grade D	all allowed	Azure Front Door
611	berkeley.edu	Score 70 out of 100, grade C	all allowed	WordPress . Cloudflare
836	academia.edu	Score 48 out of 100, grade D, partial assessment	6 blocked	unidentified
910	princeton.edu	Score 57 out of 100, grade D, partial assessment	all allowed	Drupal . Cloudflare
914	washington.edu	Score 60 out of 100, grade C	all allowed	WordPress
1050	columbia.edurefused GPTBot	Score 78 out of 100, grade B, partial assessment	all allowed	Cloudflare
1055	umich.edurefused GPTBot	Score 84 out of 100, grade B, partial assessment	all allowed	Cloudflare
1163	yale.edu	Not scored	all allowed	unidentified
1214	upenn.edu	Score 63 out of 100, grade C	all allowed	Drupal . Fastly
1289	psu.edu	Score 67 out of 100, grade C	all allowed	Contentful . Amazon CloudFront
1455	ucla.edu	Score 64 out of 100, grade C	all allowed	Amazon CloudFront
1528	wisc.edu	Score 63 out of 100, grade C	all allowed	Amazon CloudFront
1631	uci.edu	Score 57 out of 100, grade D	all allowed	unidentified
1793	cmu.edu	Score 63 out of 100, grade C	all allowed	Drupal . Fastly
1869	si.edurefused GPTBot	Score 84 out of 100, grade B, partial assessment	all allowed	Cloudflare
1871	utexas.edu	Score 57 out of 100, grade D	all allowed	Drupal . Fastly
1874	uchicago.edu	Score 64 out of 100, grade C	all allowed	Sitecore . Azure Front Door
1878	ucsd.edu	Score 58 out of 100, grade D	all allowed	unidentified
1998	jhu.edurefused GPTBot	Score 78 out of 100, grade B, partial assessment	all allowed	Cloudflare
2098	ufl.edu	Score 80 out of 100, grade B	all allowed	WordPress
2131	asu.edu	Score 60 out of 100, grade C	all allowed	Drupal . Cloudflare
2177	unc.edu	Score 80 out of 100, grade B	all allowed	WordPress . Fastly
2183	uiuc.edu	Score 77 out of 100, grade B	all allowed	WordPress
2208	duke.edu	Score 74 out of 100, grade C	all allowed	WordPress
2216	purdue.edu	Score 61 out of 100, grade C	all allowed	WordPress
2223	nyu.edu	Score 78 out of 100, grade B, partial assessment	all allowed	Amazon CloudFront
2312	umd.edu	Score 61 out of 100, grade C	3 blocked	Cloudflare
2314	msu.edu	Score 73 out of 100, grade C	all allowed	Next.js . Netlify
2379	usc.edu	Score 75 out of 100, grade B	all allowed	WordPress . Fastly
2402	ucdavis.edu	Score 60 out of 100, grade C	all allowed	Drupal . Fastly
2527	arizona.edu	Score 69 out of 100, grade C	all allowed	Drupal . Fastly
2560	illinois.edu	Score 77 out of 100, grade B	all allowed	WordPress
2602	northwestern.edu	Score 60 out of 100, grade C	all allowed	unidentified
2613	colorado.edu	Score 60 out of 100, grade C	all allowed	Drupal . Fastly
2842	virginia.edurefused GPTBot	Score 57 out of 100, grade D	all allowed	Drupal
2854	utah.edu	Score 60 out of 100, grade C	all allowed	WordPress
2944	ucsb.edu	Score 57 out of 100, grade D	all allowed	Drupal . Fastly
2956	tamu.edu	Score 64 out of 100, grade C	all allowed	WordPress . Amazon CloudFront
3011	bu.edu	Score 66 out of 100, grade C	all allowed	WordPress . Amazon CloudFront
3015	osu.edurefused GPTBot	Score 78 out of 100, grade B, partial assessment	all allowed	unidentified
3165	rutgers.edu	Score 60 out of 100, grade C	all allowed	Drupal . Varnish
3187	ncsu.edu	Score 60 out of 100, grade C	all allowed	WordPress . Amazon CloudFront
3216	pitt.edu	Score 57 out of 100, grade D	all allowed	Drupal . Fastly
3306	caltech.edu	Score 92 out of 100, grade A, partial assessment	all allowed	Cloudflare
3308	brookings.edu	Score 70 out of 100, grade C	all allowed	HubSpot CMS . Cloudflare
3311	vt.edurefused GPTBot	Score 53 out of 100, grade D	all allowed	Adobe Experience Manager . Amazon CloudFront
3466	indiana.edu	Score 57 out of 100, grade D	all allowed	unidentified
3517	georgetown.edu	Score 84 out of 100, grade B, partial assessment	all allowed	Fastly
3575	cuny.edu	Score 14 out of 100, grade F, partial assessment	11 blocked	unidentified
3580	wustl.edu	Score 80 out of 100, grade B	all allowed	WordPress . Fastly
3632	hawaii.edu	Not scored	all allowed	unidentified
3708	tufts.edu	Score 67 out of 100, grade C	all allowed	Drupal . Fastly
3748	gatech.edu	Score 60 out of 100, grade C	all allowed	Drupal . Fastly
3795	uga.edu	Score 92 out of 100, grade A, partial assessment	all allowed	Amazon CloudFront
3804	unl.edu	Score 60 out of 100, grade C	all allowed	Drupal . Varnish
3843	brown.edu	Score 63 out of 100, grade C	all allowed	Drupal . Cloudflare
3862	fsu.edu	Score 60 out of 100, grade C	all allowed	Drupal . Fastly
3871	iastate.edu	Score 60 out of 100, grade C	all allowed	Drupal
3872	hbs.edu	Score 64 out of 100, grade C	all allowed	Next.js . Amazon CloudFront
3894	rochester.edu	Score 60 out of 100, grade C	all allowed	unidentified
3949	oregonstate.edurefused GPTBot	Score 53 out of 100, grade D	all allowed	Drupal . Cloudflare
4041	uiowa.edu	Score 66 out of 100, grade C	all allowed	Drupal . Varnish
4115	monash.edurefused GPTBot	Score 78 out of 100, grade B, partial assessment	all allowed	Cloudflare
4226	umass.edurefused GPTBot	Score 78 out of 100, grade B, partial assessment	all allowed	Cloudflare
4237	iu.edu	Score 57 out of 100, grade D	all allowed	unidentified
4310	usf.edu	Score 62 out of 100, grade C	all allowed	unidentified
4405	udel.edu	Score 60 out of 100, grade C	all allowed	Adobe Experience Manager
4427	uoregon.edu	Score 64 out of 100, grade C	all allowed	Cloudflare
4433	ucsf.edurefused GPTBot	Score 78 out of 100, grade B, partial assessment	all allowed	Cloudflare
4438	vanderbilt.edu	Score 63 out of 100, grade C	all allowed	WordPress . Amazon CloudFront
4642	nd.edu	Score 66 out of 100, grade C	all allowed	unidentified
4695	wsu.edu	Score 64 out of 100, grade C	all allowed	WordPress
4737	colostate.edu	Score 77 out of 100, grade B	all allowed	WordPress . Cloudflare
4846	ucsc.edurefused GPTBot	Score 78 out of 100, grade B, partial assessment	all allowed	Cloudflare
4900	byu.edu	Score 60 out of 100, grade C	all allowed	Amazon CloudFront
4938	gmu.edurefused GPTBot	Score 53 out of 100, grade D	all allowed	Drupal . Fastly
5102	ucr.edu	Score 60 out of 100, grade C	all allowed	Drupal . Fastly

Using these figures

Free to reuse in research, journalism or a product under CC BY 4.0, with credit to Fidget Labs BV. Quote the measurement date so the claim stays checkable as the index moves.

CrawlIndex by Fidget Labs BV. "Agent readiness on .edu." https://crawlindex.org (measured 2026-09-22). Licensed CC BY 4.0.