Top-level domain

.edu

Name: CrawlIndex dataset
License: https://creativecommons.org/licenses/by/4.0/

81 measured sites. 5 of them, 6.2%, block at least one answer-surface AI crawler. Mean readiness score 66.4.

Compare against every other top-level domain

Sites on .edu
Rank	Domain	Score	Answer-surface crawlers	Stack
220	mit.edu	Score 62 out of 100, grade C	all allowed	Akamai
296	harvard.edu	Score 71 out of 100, grade C	all allowed	WordPress
363	stanford.edu	Score 67 out of 100, grade C	all allowed	Storyblok . Netlify
546	cornell.edurefused GPTBot	Score 55 out of 100, grade D	all allowed	Azure Front Door
560	berkeley.edu	Score 72 out of 100, grade C	all allowed	WordPress . Cloudflare
793	washington.edu	Score 62 out of 100, grade C	all allowed	WordPress
815	academia.edu	Score 53 out of 100, grade D, partial assessment	6 blocked	unidentified
860	princeton.edu	Score 59 out of 100, grade D, partial assessment	all allowed	Drupal . Cloudflare
951	umich.edurefused GPTBot	Score 89 out of 100, grade B, partial assessment	all allowed	Cloudflare
994	columbia.edurefused GPTBot	Score 83 out of 100, grade B, partial assessment	all allowed	Cloudflare
1036	yale.edu	Score 59 out of 100, grade D	all allowed	Drupal . Fastly
1135	upenn.edu	Score 62 out of 100, grade C	all allowed	Drupal . Fastly
1206	psu.edu	Score 68 out of 100, grade C	all allowed	Contentful . Amazon CloudFront
1323	wisc.edu	Score 66 out of 100, grade C	all allowed	Amazon CloudFront
1407	uci.edu	Score 59 out of 100, grade D	all allowed	unidentified
1430	ucla.edu	Score 67 out of 100, grade C	all allowed	Amazon CloudFront
1500	umn.edu	Score 70 out of 100, grade C	all allowed	Drupal . Cloudflare
1568	cmu.edu	Score 66 out of 100, grade C	all allowed	Drupal . Fastly
1823	ucsd.edu	Score 62 out of 100, grade C	all allowed	unidentified
1872	uchicago.edurefused GPTBot	Score 83 out of 100, grade B, partial assessment	all allowed	Azure Front Door
1880	utexas.edu	Score 59 out of 100, grade D	all allowed	Drupal . Fastly
2006	jhu.edurefused GPTBot	Score 55 out of 100, grade D, partial assessment	5 blocked	Cloudflare
2062	si.edurefused GPTBot	Score 55 out of 100, grade D, partial assessment	5 blocked	Cloudflare
2195	ufl.edu	Score 83 out of 100, grade B	all allowed	WordPress
2252	purdue.edu	Score 64 out of 100, grade C	all allowed	WordPress
2254	duke.edu	Score 80 out of 100, grade B	all allowed	WordPress
2269	nyu.edu	Not scored	all allowed	unidentified
2280	uiuc.edu	Score 83 out of 100, grade B	all allowed	WordPress
2320	unc.edu	Score 83 out of 100, grade B	all allowed	WordPress . Fastly
2399	asu.edu	Score 63 out of 100, grade C	all allowed	Drupal . Cloudflare
2414	umd.edu	Score 63 out of 100, grade C	3 blocked	Cloudflare
2442	usc.edu	Score 75 out of 100, grade B	all allowed	WordPress . Fastly
2471	msu.edu	Score 67 out of 100, grade C	all allowed	Next.js . Netlify
2473	ucdavis.edu	Score 62 out of 100, grade C	all allowed	Drupal . Fastly
2530	colorado.edu	Score 62 out of 100, grade C	all allowed	Drupal . Fastly
2569	illinois.edu	Score 83 out of 100, grade B	all allowed	WordPress
2598	arizona.edu	Score 70 out of 100, grade C	all allowed	Drupal . Fastly
2630	northwestern.edu	Score 62 out of 100, grade C	all allowed	unidentified
2793	utah.edu	Score 62 out of 100, grade C	all allowed	WordPress
2823	osu.edurefused GPTBot	Score 83 out of 100, grade B, partial assessment	all allowed	unidentified
2833	virginia.edurefused GPTBot	Score 60 out of 100, grade C	all allowed	Drupal
2863	bu.edu	Score 70 out of 100, grade C	all allowed	WordPress . Amazon CloudFront
2887	tamu.edu	Score 67 out of 100, grade C	all allowed	WordPress . Amazon CloudFront
2982	rutgers.edu	Score 63 out of 100, grade C	all allowed	Drupal . Varnish
3029	ucsb.edu	Score 59 out of 100, grade D	all allowed	Drupal . Fastly
3069	ncsu.edu	Score 60 out of 100, grade C	all allowed	WordPress . Amazon CloudFront
3102	brookings.edu	Score 72 out of 100, grade C	all allowed	HubSpot CMS . Cloudflare
3200	pitt.edu	Score 59 out of 100, grade D	all allowed	Drupal . Fastly
3230	caltech.edu	Score 100 out of 100, grade A, partial assessment	all allowed	Cloudflare
3282	iastate.edu	Score 62 out of 100, grade C	all allowed	Drupal
3320	georgetown.edu	Score 89 out of 100, grade B, partial assessment	all allowed	Fastly
3330	wustl.edu	Score 83 out of 100, grade B	all allowed	WordPress . Fastly
3338	cuny.edu	Score 17 out of 100, grade F, partial assessment	11 blocked	unidentified
3362	vt.edurefused GPTBot	Score 55 out of 100, grade D	all allowed	Adobe Experience Manager . Amazon CloudFront
3439	hawaii.edu	Score 62 out of 100, grade C	all allowed	unidentified
3446	indiana.edu	Not scored	all allowed	unidentified
3512	uga.edu	Score 53 out of 100, grade D	all allowed	Amazon CloudFront
3522	gatech.edurefused GPTBot	Score 55 out of 100, grade D	all allowed	Drupal . Amazon CloudFront
3545	tufts.edu	Score 72 out of 100, grade C	all allowed	Drupal . Fastly
3591	unl.edu	Score 62 out of 100, grade C	all allowed	Drupal . Varnish
3606	hbs.edu	Score 67 out of 100, grade C	all allowed	Next.js . Amazon CloudFront
3656	rochester.edu	Score 62 out of 100, grade C	all allowed	unidentified
3666	oregonstate.edu	Score 62 out of 100, grade C	all allowed	Drupal . Varnish
3745	brown.edu	Score 62 out of 100, grade C	all allowed	Drupal . Cloudflare
3970	uiowa.edu	Score 67 out of 100, grade C	all allowed	Drupal . Varnish
4070	iu.edu	Not scored	all allowed	unidentified
4085	umass.edu	Score 67 out of 100, grade C	all allowed	Drupal . Cloudflare
4111	uoregon.edu	Score 59 out of 100, grade D	all allowed	Drupal . Fastly
4181	gwu.edu	Not scored	all allowed	unidentified
4184	udel.edu	Score 62 out of 100, grade C	all allowed	Adobe Experience Manager
4207	usf.edu	Score 62 out of 100, grade C	all allowed	unidentified
4253	fsu.edu	Score 62 out of 100, grade C	all allowed	unidentified
4296	vanderbilt.edu	Score 60 out of 100, grade C	all allowed	WordPress . Amazon CloudFront
4301	ucsf.edu	Score 79 out of 100, grade B	all allowed	Drupal . Cloudflare
4398	ucsc.edurefused GPTBot	Score 83 out of 100, grade B, partial assessment	all allowed	Cloudflare
4506	colostate.edurefused GPTBot	Score 73 out of 100, grade C	all allowed	WordPress . Cloudflare
4548	wsu.edu	Score 67 out of 100, grade C	all allowed	WordPress
4586	monash.edurefused GPTBot	Score 83 out of 100, grade B, partial assessment	all allowed	Cloudflare
4664	nd.edu	Score 67 out of 100, grade C	all allowed	unidentified
4731	byu.edu	Score 62 out of 100, grade C	all allowed	Amazon CloudFront
4851	emory.edu	Score 62 out of 100, grade C	all allowed	Drupal
4854	uw.edu	Score 62 out of 100, grade C	all allowed	WordPress
4881	rice.edu	Not scored	all allowed	unidentified
4994	gmu.edu	Score 62 out of 100, grade C	all allowed	Drupal . Fastly
5069	dartmouth.edu	Score 59 out of 100, grade D	all allowed	Drupal . Cloudflare
5102	ucr.edu	Score 60 out of 100, grade C	all allowed	Drupal . Fastly

Using these figures

Free to reuse in research, journalism or a product under CC BY 4.0, with credit to Fidget Labs BV. Quote the measurement date so the claim stays checkable as the index moves.

CrawlIndex by Fidget Labs BV. "Agent readiness on .edu." https://crawlindex.org (measured 2026-08-09). Licensed CC BY 4.0.