crawlindex

Search the index. A domain that is not here can still be measured live on the check page.

Top-level domain

.com

1,946 measured sites. 360 of them, 18.5%, block at least one answer-surface AI crawler. Mean readiness score 65.8.

Compare against every other top-level domain

Sites on .com
RankDomainScoreAnswer-surface crawlersStack
1google.comrefused GPTBotScore 52 out of 100, grade Dall allowedGoogle Cloud / GFE
2cloudflare.comScore 85 out of 100, grade Ball allowedAstro . Cloudflare
5facebook.comScore 6 out of 100, grade F, partial assessment11 blockedunidentified
9microsoft.comScore 62 out of 100, grade Call allowedAdobe Experience Manager . Azure Front Door
11youtube.comScore 56 out of 100, grade Dall allowedGoogle Cloud / GFE
12apple.comScore 77 out of 100, grade Ball allowedunidentified
14instagram.comScore 14 out of 100, grade F, partial assessment11 blockedunidentified
16twitter.comScore 14 out of 100, grade F, partial assessment11 blockedunidentified
17linkedin.comScore 17 out of 100, grade F, partial assessment9 blockedunidentified
23triplinkintl.comScore 57 out of 100, grade Dall allowedunidentified
27appsflyersdk.comrefused GPTBotScore 78 out of 100, grade B, partial assessmentall allowedAmazon CloudFront
30shiabank.comScore 54 out of 100, grade Dall allowedunidentified
31amazon.comrefused GPTBotScore 15 out of 100, grade F10 blockedAmazon CloudFront
34github.comScore 72 out of 100, grade Call allowedContentful
35sharepoint.comScore 55 out of 100, grade Dall allowedAdobe Experience Manager . Azure Front Door
43netflix.comScore 59 out of 100, grade D, partial assessment3 blockedunidentified
44lsrelayaccess.comScore 78 out of 100, grade B, partial assessmentall allowedunidentified
46bing.comScore 59 out of 100, grade Dall allowedunidentified
47x.comScore 14 out of 100, grade F, partial assessment11 blockedunidentified
52pinterest.comScore 17 out of 100, grade F, partial assessment11 blockedunidentified
56icloud.comScore 52 out of 100, grade Dall allowedunidentified
62adobe.comScore 92 out of 100, grade A, partial assessmentall allowedAkamai
63tiktok.comScore 15 out of 100, grade F11 blockedunidentified
68googledomains.comScore 52 out of 100, grade Dall allowedunidentified
71yahoo.comrefused GPTBotScore 43 out of 100, grade D5 blockedNext.js
73digicert.comScore 92 out of 100, grade A, partial assessmentall allowedImperva
77opera.comScore 85 out of 100, grade Ball allowedunidentified
78spotify.comScore 52 out of 100, grade Dall allowedFastly
79roblox.comScore 61 out of 100, grade Call allowedunidentified
83amazonvideo.comScore 40 out of 100, grade D5 blockedAmazon CloudFront
85baidu.comScore 6 out of 100, grade F, partial assessment11 blockedunidentified
87vimeo.comScore 56 out of 100, grade D3 blockedNext.js . Cloudflare
88blogspot.comScore 57 out of 100, grade Dall allowedunidentified
90qq.comScore 59 out of 100, grade Dall allowedunidentified
96epicgames.comrefused GPTBotScore 78 out of 100, grade B, partial assessmentall allowedCloudflare
98f5.comScore 72 out of 100, grade Call allowedSanity . Vercel
99wordpress.comrefused GPTBotScore 77 out of 100, grade Ball allowedWordPress
101nginx.comScore 61 out of 100, grade Call allowedNext.js . Vercel
103reddit.comScore 6 out of 100, grade F, partial assessment11 blockedunidentified
108ui.comScore 74 out of 100, grade Call allowedAmazon CloudFront
112chatgpt.comScore 14 out of 100, grade F, partial assessment11 blockedunidentified
115intuit.comScore 92 out of 100, grade A, partial assessmentall allowedAkamai
122samsung.comScore 86 out of 100, grade Ball allowedAdobe Experience Manager . Akamai
130openai.comrefused GPTBotScore 57 out of 100, grade Dall allowedContentful . Cloudflare
132android.comScore 67 out of 100, grade Call allowedunidentified
134dropbox.comScore 83 out of 100, grade Ball allowedunidentified
137outlook.comScore 45 out of 100, grade Dall allowedunidentified
144webex.comScore 78 out of 100, grade B, partial assessmentall allowedAkamai
156vk.comScore 48 out of 100, grade Dall allowedunidentified
157crashlytics.comScore 60 out of 100, grade Call allowedGoogle Cloud / GFE
158snapchat.comScore 46 out of 100, grade D5 blockedNext.js . Google Cloud / GFE
159tumblr.comScore 41 out of 100, grade D4 blockedWordPress
161godaddy.comScore 78 out of 100, grade B, partial assessmentall allowedAkamai
163nytimes.comrefused GPTBotScore 21 out of 100, grade F, partial assessment11 blockedFastly
167gravatar.comScore 56 out of 100, grade D2 blockedWordPress
168ubuntu.comScore 83 out of 100, grade Ball allowedunidentified
170amazon-adsystem.comrefused GPTBotScore 78 out of 100, grade B, partial assessmentall allowedunidentified
171shopify.comScore 73 out of 100, grade Call allowedShopify . Cloudflare
172kaspersky.comScore 86 out of 100, grade Ball allowedNext.js
174samsungcloud.comScore 52 out of 100, grade Dall allowedNext.js . Amazon CloudFront
175paypal.comrefused GPTBotScore 79 out of 100, grade Ball allowedFastly
180flickr.comScore 42 out of 100, grade D, partial assessment6 blockedunidentified
184discord.comScore 77 out of 100, grade Ball allowedWebflow . Cloudflare
187meraki.comScore 78 out of 100, grade B, partial assessmentall allowedAkamai
188ebay.comScore 60 out of 100, grade C, partial assessment5 blockedAkamai
189unity3d.comScore 79 out of 100, grade Ball allowedSanity
191soundcloud.comScore 31 out of 100, grade F, partial assessment10 blockedAmazon CloudFront
193cnn.comScore 31 out of 100, grade F9 blockedFastly
196theguardian.comScore 46 out of 100, grade D6 blockedunidentified
202medium.comrefused GPTBotScore 46 out of 100, grade D4 blockedCloudflare
205afternic.comScore 60 out of 100, grade Call allowedContentful . Akamai
206stripe.comScore 80 out of 100, grade Ball allowedNext.js
207forter.comScore 87 out of 100, grade B1 blockedHubSpot CMS . Cloudflare
213oracle.comScore 78 out of 100, grade B, partial assessmentall allowedAkamai
214bybanking.comScore 54 out of 100, grade Dall allowedunidentified
218bbc.comScore 31 out of 100, grade F10 blockedNext.js . Fastly
220firefox-portal-detection.comScore 78 out of 100, grade B, partial assessmentall allowedFastly
222xiaomi.comScore 60 out of 100, grade Call allowedAkamai
223arubanetworks.comScore 78 out of 100, grade B, partial assessmentall allowedAkamai
226publicnode.comScore 59 out of 100, grade Dall allowedNext.js . Cloudflare
228hcaptcha.comScore 81 out of 100, grade Ball allowedCloudflare
229forbes.comrefused GPTBotScore 57 out of 100, grade D, partial assessment5 blockedFastly
233sciencedirect.comrefused GPTBotScore 78 out of 100, grade B, partial assessmentall allowedCloudflare
236applovin.comScore 79 out of 100, grade Ball allowedNext.js . Google Cloud / GFE
238weather.comScore 82 out of 100, grade Ball allowedNext.js . Akamai
239roku.comScore 77 out of 100, grade Ball allowedNext.js . Cloudflare
242namecheap.comrefused GPTBotScore 92 out of 100, grade A, partial assessmentall allowedCloudflare
251cisco.comScore 92 out of 100, grade A, partial assessmentall allowedAkamai
255icanhazip.comScore 84 out of 100, grade B, partial assessmentall allowedCloudflare
259dnsmadeeasy.comScore 55 out of 100, grade Dall allowedAdobe Experience Manager . Fastly
260hubspot.comScore 82 out of 100, grade Ball allowedHubSpot CMS . Cloudflare
263imdb.comScore 9 out of 100, grade F, partial assessment11 blockedunidentified
266alidns.comScore 55 out of 100, grade Dall allowedAlibaba Cloud CDN
268googleblog.comScore 57 out of 100, grade Dall allowedGoogle Cloud / GFE
271brave.comScore 75 out of 100, grade Ball allowedAmazon CloudFront
277ibm.comScore 73 out of 100, grade C, partial assessmentall allowedAdobe Experience Manager . Akamai
280canva.comrefused GPTBotScore 66 out of 100, grade C, partial assessment4 blockedCloudflare
281userapi.comScore 48 out of 100, grade Dall allowedunidentified
283reuters.comScore 26 out of 100, grade F, partial assessment9 blockedunidentified
285hp.comScore 62 out of 100, grade Call allowedAdobe Experience Manager . Akamai
288gamepass.comScore 52 out of 100, grade Dall allowedunidentified
291booking.comScore 92 out of 100, grade A, partial assessmentall allowedAmazon CloudFront
293nature.comScore 45 out of 100, grade D, partial assessment7 blockedFastly
296ip-api.comScore 60 out of 100, grade Call allowedunidentified
298steampowered.comScore 55 out of 100, grade Dall allowedunidentified
302salesforce.comScore 77 out of 100, grade Ball allowedContentful . Akamai
304slack.comScore 75 out of 100, grade Ball allowedunidentified
305miui.comNot scoredall allowedunidentified
306launchdarkly.comScore 73 out of 100, grade Call allowedNext.js . Netlify
307springer.comScore 92 out of 100, grade A, partial assessmentall allowedFastly
309alibaba.comScore 53 out of 100, grade D2 blockedAlibaba Cloud CDN
314avast.comScore 62 out of 100, grade Call allowedAdobe Experience Manager . Fastly
317vungle.comrefused GPTBotScore 69 out of 100, grade Call allowedWordPress . Cloudflare
320autodesk.comScore 78 out of 100, grade B, partial assessmentall allowedAkamai
323grammarly.comScore 64 out of 100, grade Call allowedContentful . Amazon CloudFront
325weibo.comScore 18 out of 100, grade F, partial assessment11 blockedunidentified
328wiley.comrefused GPTBotScore 47 out of 100, grade Dall allowedCloudflare
334etsy.comrefused GPTBotScore 84 out of 100, grade B, partial assessmentall allowedFastly
335anthropic.comScore 64 out of 100, grade Call allowedSanity . Cloudflare
336mi.comScore 60 out of 100, grade Call allowedAkamai
337unisys.comScore 83 out of 100, grade Ball allowedCloudflare
338gmail.comScore 64 out of 100, grade Call allowedGoogle Cloud / GFE
341zendesk.comScore 89 out of 100, grade Ball allowedNext.js . Cloudflare
343smartadserver.comScore 55 out of 100, grade Dall allowedWordPress . Cloudflare
344naver.comScore 6 out of 100, grade F, partial assessment11 blockedunidentified
347browser-intake-datadoghq.comrefused GPTBotScore 78 out of 100, grade B, partial assessmentall allowedunidentified
348bilibili.comrefused GPTBotScore 57 out of 100, grade Dall allowedunidentified
350ea.comScore 66 out of 100, grade C1 blockedNext.js
351tp-link.comScore 74 out of 100, grade Call allowedAmazon CloudFront
353trustpilot.comScore 66 out of 100, grade C, partial assessment2 blockedunidentified
355espn.comScore 72 out of 100, grade C, partial assessment3 blockedAmazon CloudFront
357weebly.comScore 69 out of 100, grade Call allowedCloudflare
358aliyun.comScore 55 out of 100, grade Dall allowedAlibaba Cloud CDN
360checkpoint.comScore 77 out of 100, grade Ball allowedWordPress . Amazon CloudFront
361dell.comScore 80 out of 100, grade Ball allowedAdobe Experience Manager . Akamai
362tradingview.comScore 74 out of 100, grade Call allowedAmazon CloudFront
363nvidia.comScore 82 out of 100, grade Ball allowedAdobe Experience Manager
366cm.comScore 81 out of 100, grade Ball allowedCloudflare
367paloaltonetworks.comScore 71 out of 100, grade Call allowedAkamai
370globo.comScore 56 out of 100, grade D2 blockedunidentified
372wsj.comScore 31 out of 100, grade F, partial assessment8 blockedunidentified
373dotomi.comScore 69 out of 100, grade Call allowedNext.js . Amazon CloudFront
374opendns.comScore 58 out of 100, grade Dall allowedunidentified
376gitlab.comScore 73 out of 100, grade Call allowedNuxt . Cloudflare
378tagomi.comrefused GPTBotScore 78 out of 100, grade B, partial assessmentall allowedCloudflare
383issuu.comScore 64 out of 100, grade Call allowedNext.js . Fastly
388bloomberg.comrefused GPTBotScore 41 out of 100, grade D10 blockedNext.js . Fastly
394okta.comScore 77 out of 100, grade Ball allowedAdobe Experience Manager . Cloudflare
396plesk.comScore 79 out of 100, grade Ball allowedWordPress . Cloudflare
399name.comScore 80 out of 100, grade B1 blockedSanity . Cloudflare
400netease.comScore 52 out of 100, grade Dall allowedunidentified
402go.comScore 56 out of 100, grade D2 blockedAmazon CloudFront
403amazontrust.comScore 52 out of 100, grade Dall allowedAmazon CloudFront
405wps.comScore 81 out of 100, grade Ball allowedNuxt . Amazon CloudFront
407yandex.comScore 56 out of 100, grade Dall allowedunidentified
415dynatrace.comScore 89 out of 100, grade Ball allowedNuxt . Amazon CloudFront
416calendly.comScore 82 out of 100, grade Ball allowedAstro . Cloudflare
419elasticbeanstalk.comScore 67 out of 100, grade Call allowedAmazon CloudFront
420xboxlive.comScore 55 out of 100, grade Dall allowedunidentified
423duckduckgo.comScore 64 out of 100, grade Call allowedNext.js
425expireddomains.comrefused GPTBotScore 84 out of 100, grade B, partial assessmentall allowedCloudflare
426163.comrefused GPTBotScore 51 out of 100, grade Dall allowedAlibaba Cloud CDN
427indeed.comrefused GPTBotScore 84 out of 100, grade B, partial assessmentall allowedCloudflare
428firetvcaptiveportal.comScore 78 out of 100, grade B, partial assessmentall allowedunidentified
429substack.comScore 78 out of 100, grade B, partial assessmentall allowedCloudflare
430aol.comScore 50 out of 100, grade D3 blockedNext.js . Amazon CloudFront
434steamcommunity.comScore 55 out of 100, grade Dall allowedunidentified
437temu.comScore 74 out of 100, grade Call allowedunidentified
445dailymotion.comScore 58 out of 100, grade Dall allowedAmazon CloudFront
451inmobi.comScore 62 out of 100, grade Call allowedNext.js . Google Cloud / GFE
453intel.comScore 92 out of 100, grade A, partial assessmentall allowedAkamai
455hostgator.comScore 78 out of 100, grade B, partial assessmentall allowedCloudflare
456dropcatch.comScore 48 out of 100, grade Dall allowedCloudflare
459taobao.comScore 55 out of 100, grade Dall allowedAlibaba Cloud CDN
464lijit.comScore 49 out of 100, grade Dall allowedAmazon CloudFront
465pixabay.comrefused GPTBotScore 67 out of 100, grade C, partial assessment4 blockedCloudflare
467duolingo.comScore 52 out of 100, grade Dall allowedAmazon CloudFront
470bluehost.comScore 78 out of 100, grade B, partial assessmentall allowedCloudflare
473huawei.comScore 71 out of 100, grade Call allowedSitecore
474mcafee.comScore 78 out of 100, grade B, partial assessmentall allowedAkamai
477businessinsider.comScore 74 out of 100, grade C1 blockedFastly
478aliexpress.comScore 71 out of 100, grade C, partial assessment3 blockedAlibaba Cloud CDN
480walmart.comScore 74 out of 100, grade Call allowedNext.js
481cnbc.comrefused GPTBotScore 24 out of 100, grade F11 blockedAkamai
483shein.comScore 71 out of 100, grade Call allowedCloudflare
484goodreads.comScore 56 out of 100, grade D1 blockedAmazon CloudFront
485wp.comrefused GPTBotScore 77 out of 100, grade Ball allowedWordPress
487fandom.comScore 78 out of 100, grade B, partial assessmentall allowedCloudflare
496trendmicro.comScore 77 out of 100, grade Ball allowedAdobe Experience Manager
497quora.comScore 13 out of 100, grade F, partial assessment10 blockedunidentified
498statista.comrefused GPTBotScore 43 out of 100, grade D7 blockedAmazon CloudFront
499mzstatic.comScore 70 out of 100, grade Call allowedunidentified
500scribd.comScore 67 out of 100, grade Call allowedNext.js . Fastly
502eventbrite.comScore 64 out of 100, grade Call allowedNext.js . Amazon CloudFront
504teamviewer.comrefused GPTBotScore 64 out of 100, grade Call allowedNext.js . Cloudflare
507foxnews.comScore 89 out of 100, grade Ball allowedFastly
508addtoany.comScore 55 out of 100, grade Dall allowedCloudflare
513braze.comScore 73 out of 100, grade Call allowedSanity . Cloudflare
514indiatimes.comScore 74 out of 100, grade C1 blockedunidentified
515playstation.comScore 67 out of 100, grade Call allowedAdobe Experience Manager . Fastly

Showing the 200 most-visited.

Using these figures

Free to reuse in research, journalism or a product under CC BY 4.0, with credit to Fidget Labs BV. Quote the measurement date so the claim stays checkable as the index moves.

CrawlIndex by Fidget Labs BV. "Agent readiness on .com." https://crawlindex.org (measured 2026-11-07). Licensed CC BY 4.0.