crawlindex

Search the index. A domain that is not here can still be measured live on the check page.

Top-level domain

.gov

71 measured sites. 4 of them, 5.6%, block at least one answer-surface AI crawler. Mean readiness score 65.8.

Compare against every other top-level domain

Sites on .gov
RankDomainScoreAnswer-surface crawlersStack
130nih.govrefused GPTBotScore 78 out of 100, grade B, partial assessmentall allowedCloudflare
236nist.govScore 67 out of 100, grade Call allowedDrupal . Cloudflare
368cdc.govScore 92 out of 100, grade A, partial assessmentall allowedAkamai
410nasa.govScore 75 out of 100, grade Ball allowedWordPress
576ca.govScore 85 out of 100, grade Ball allowedAzure Front Door
578noaa.govScore 60 out of 100, grade Call allowedDrupal . Amazon CloudFront
583maricopa.govrefused GPTBotScore 54 out of 100, grade Dall allowedCloudflare
656loc.govrefused GPTBotScore 57 out of 100, grade Dall allowedCloudflare
863usda.govScore 78 out of 100, grade B, partial assessmentall allowedAkamai
899epa.govScore 63 out of 100, grade Call allowedDrupal . Amazon CloudFront
943state.govrefused GPTBotScore 53 out of 100, grade Dall allowedWordPress . Amazon CloudFront
1031whitehouse.govScore 80 out of 100, grade Ball allowedWordPress
1331irs.govScore 61 out of 100, grade Call allowedDrupal . Akamai
1798va.govScore 76 out of 100, grade Ball allowedNext.js
1880nps.govScore 61 out of 100, grade Call allowedAmazon CloudFront
1915sec.govScore 92 out of 100, grade A, partial assessmentall allowedAkamai
2073bls.govScore 78 out of 100, grade B, partial assessmentall allowedAkamai
2149ed.govrefused GPTBotScore 78 out of 100, grade B, partial assessmentall allowedAzure Front Door
2177house.govScore 57 out of 100, grade Dall allowedDrupal . Akamai
2181hhs.govScore 78 out of 100, grade B, partial assessmentall allowedunidentified
2201census.govScore 69 out of 100, grade Call allowedAdobe Experience Manager . Cloudflare
2203justice.govScore 52 out of 100, grade D3 blockedDrupal . Akamai
2348ny.govScore 92 out of 100, grade A, partial assessmentall allowedCloudflare
2371weather.govScore 55 out of 100, grade Dall allowedunidentified
2383energy.govScore 66 out of 100, grade Call allowedDrupal . Amazon CloudFront
2427congress.govrefused GPTBotScore 21 out of 100, grade F, partial assessment10 blockedCloudflare
2451faa.govScore 57 out of 100, grade Dall allowedDrupal
2477senate.govScore 52 out of 100, grade Dall allowedAkamai
2547dhs.govrefused GPTBotScore 57 out of 100, grade Dall allowedDrupal . Akamai
2734medlineplus.govScore 74 out of 100, grade Call allowedAmazon CloudFront
2743archives.govScore 63 out of 100, grade Call allowedAmazon CloudFront
2820treasury.govScore 57 out of 100, grade Dall allowedAkamai
2832texas.govScore 77 out of 100, grade Ball allowedDrupal . Amazon CloudFront
2861wa.govScore 14 out of 100, grade F, partial assessment11 blockedunidentified
2893dot.govScore 78 out of 100, grade B, partial assessmentall allowedAkamai
2964fcc.govrefused GPTBotScore 78 out of 100, grade B, partial assessmentall allowedAkamai
3018uspto.govScore 80 out of 100, grade Ball allowedDrupal . Amazon CloudFront
3086nyc.govScore 63 out of 100, grade Call allowedAdobe Experience Manager . Fastly
3113fbi.govrefused GPTBotScore 78 out of 100, grade B, partial assessmentall allowedCloudflare
3117usa.govScore 74 out of 100, grade Call allowedDrupal . Amazon CloudFront
3149cancer.govScore 64 out of 100, grade Call allowedDrupal
3200federalregister.govScore 62 out of 100, grade Call allowedunidentified
3212dol.govScore 84 out of 100, grade B, partial assessmentall allowedAkamai
3254dataprivacyframework.govScore 84 out of 100, grade B, partial assessmentall allowedunidentified
3302cisa.govrefused GPTBotScore 53 out of 100, grade Dall allowedDrupal
3518osha.govScore 54 out of 100, grade Dall allowedDrupal . Amazon CloudFront
3617clinicaltrials.govScore 56 out of 100, grade Dall allowedGoogle Cloud / GFE
3627federalreserve.govScore 57 out of 100, grade Dall allowedCloudflare
3642sba.govScore 80 out of 100, grade Ball allowedWordPress . Amazon CloudFront
3748uscourts.govScore 63 out of 100, grade Call allowedDrupal . Akamai
3910uscis.govrefused GPTBotScore 53 out of 100, grade Dall allowedDrupal
3922fema.govrefused GPTBotScore 78 out of 100, grade B, partial assessment1 blockedAkamai
3936cia.govScore 61 out of 100, grade Call allowedGatsby . Akamai
3950virginia.govScore 59 out of 100, grade Dall allowedunidentified
3984login.govScore 64 out of 100, grade Call allowedAmazon CloudFront
3994eia.govScore 64 out of 100, grade Call allowedAkamai
4061studentaid.govScore 45 out of 100, grade Dall allowedDrupal . Akamai
4192cms.govScore 64 out of 100, grade Call allowedDrupal . Akamai
4338govinfo.govScore 61 out of 100, grade Call allowedDrupal . Cloudflare
4436defense.govrefused GPTBotScore 78 out of 100, grade B, partial assessmentall allowedAkamai
4462nj.govScore 57 out of 100, grade Dall allowedImperva
4638mo.govScore 60 out of 100, grade Call allowedDrupal . Imperva
4653ct.govScore 57 out of 100, grade Dall allowedSitecore . Azure Front Door
4658nsf.govScore 67 out of 100, grade Call allowedDrupal . Amazon CloudFront
4680in.govScore 92 out of 100, grade A, partial assessmentall allowedunidentified
4767ecfr.govScore 62 out of 100, grade Call allowedunidentified
4809pa.govScore 64 out of 100, grade Call allowedAdobe Experience Manager . Fastly
4976nhtsa.govScore 78 out of 100, grade B, partial assessmentall allowedAkamai
5017cbp.govrefused GPTBotScore 53 out of 100, grade Dall allowedDrupal
5065medicare.govScore 63 out of 100, grade Call allowedDrupal . Akamai
5075utah.govScore 54 out of 100, grade Dall allowedAmazon CloudFront

Using these figures

Free to reuse in research, journalism or a product under CC BY 4.0, with credit to Fidget Labs BV. Quote the measurement date so the claim stays checkable as the index moves.

CrawlIndex by Fidget Labs BV. "Agent readiness on .gov." https://crawlindex.org (measured 2026-11-22). Licensed CC BY 4.0.