crawlindex

Search the index. A domain that is not here can still be measured live on the check page.

Top-level domain

.com

1,970 measured sites. 361 of them, 18.3%, block at least one answer-surface AI crawler. Mean readiness score 66.

Compare against every other top-level domain

Sites on .com
RankDomainScoreAnswer-surface crawlersStack
1google.comrefused GPTBotScore 49 out of 100, grade Dall allowedGoogle Cloud / GFE
2cloudflare.comScore 85 out of 100, grade Ball allowedAstro . Cloudflare
5facebook.comScore 6 out of 100, grade F, partial assessment11 blockedunidentified
9microsoft.comScore 65 out of 100, grade Call allowedAdobe Experience Manager . Azure Front Door
10youtube.comScore 56 out of 100, grade Dall allowedGoogle Cloud / GFE
12apple.comScore 77 out of 100, grade Ball allowedunidentified
14instagram.comScore 14 out of 100, grade F, partial assessment11 blockedunidentified
16twitter.comScore 14 out of 100, grade F, partial assessment11 blockedunidentified
17linkedin.comScore 17 out of 100, grade F, partial assessment9 blockedunidentified
19triplinkintl.comScore 57 out of 100, grade Dall allowedunidentified
24appsflyersdk.comrefused GPTBotScore 78 out of 100, grade B, partial assessmentall allowedAmazon CloudFront
29amazon.comrefused GPTBotScore 16 out of 100, grade F, partial assessment10 blockedAmazon CloudFront
31sharepoint.comScore 55 out of 100, grade Dall allowedAdobe Experience Manager . Azure Front Door
34github.comScore 72 out of 100, grade Call allowedContentful
38lsrelayaccess.comScore 78 out of 100, grade B, partial assessmentall allowedunidentified
43netflix.comScore 59 out of 100, grade D, partial assessment3 blockedunidentified
46bing.comScore 61 out of 100, grade Call allowedunidentified
48x.comScore 14 out of 100, grade F, partial assessment11 blockedunidentified
55pinterest.comScore 17 out of 100, grade F, partial assessment11 blockedunidentified
56icloud.comScore 52 out of 100, grade Dall allowedunidentified
61adobe.comScore 92 out of 100, grade A, partial assessmentall allowedAkamai
63tiktok.comScore 15 out of 100, grade F11 blockedunidentified
70googledomains.comScore 52 out of 100, grade Dall allowedunidentified
72yahoo.comrefused GPTBotScore 43 out of 100, grade D5 blockedNext.js
74digicert.comScore 92 out of 100, grade A, partial assessmentall allowedImperva
78spotify.comScore 52 out of 100, grade Dall allowedFastly
79opera.comScore 85 out of 100, grade Ball allowedunidentified
80epicgames.comrefused GPTBotScore 78 out of 100, grade B, partial assessmentall allowedCloudflare
81amazonvideo.comScore 40 out of 100, grade D5 blockedAmazon CloudFront
82roblox.comScore 61 out of 100, grade Call allowedunidentified
84vimeo.comScore 56 out of 100, grade D3 blockedNext.js . Cloudflare
90baidu.comScore 6 out of 100, grade F, partial assessment11 blockedunidentified
91qq.comScore 55 out of 100, grade Dall allowedunidentified
92blogspot.comScore 57 out of 100, grade Dall allowedunidentified
98intuit.comScore 92 out of 100, grade A, partial assessmentall allowedAkamai
99f5.comScore 72 out of 100, grade Call allowedSanity . Vercel
100wordpress.comrefused GPTBotScore 77 out of 100, grade Ball allowedWordPress
104nginx.comScore 61 out of 100, grade Call allowedNext.js . Vercel
110ui.comScore 74 out of 100, grade Call allowedNext.js . Amazon CloudFront
111reddit.comScore 6 out of 100, grade F, partial assessment11 blockedunidentified
114chatgpt.comScore 14 out of 100, grade F, partial assessment11 blockedunidentified
123samsung.comScore 86 out of 100, grade Ball allowedAdobe Experience Manager . Akamai
124webex.comScore 78 out of 100, grade B, partial assessmentall allowedAkamai
125openai.comScore 64 out of 100, grade Call allowedContentful . Cloudflare
132dropbox.comScore 83 out of 100, grade Ball allowedunidentified
133android.comScore 67 out of 100, grade Call allowedunidentified
137outlook.comScore 45 out of 100, grade Dall allowedunidentified
153tumblr.comScore 54 out of 100, grade D4 blockedWordPress
154crashlytics.comScore 60 out of 100, grade Call allowedGoogle Cloud / GFE
155nytimes.comrefused GPTBotScore 21 out of 100, grade F, partial assessment11 blockedFastly
159snapchat.comScore 46 out of 100, grade D5 blockedNext.js . Google Cloud / GFE
160samsungcloud.comScore 52 out of 100, grade Dall allowedNext.js . Amazon CloudFront
161ubuntu.comScore 83 out of 100, grade Ball allowedunidentified
162vk.comScore 48 out of 100, grade Dall allowedunidentified
164meraki.comScore 78 out of 100, grade B, partial assessmentall allowedAkamai
167gravatar.comScore 60 out of 100, grade C2 blockedWordPress
168shopify.comScore 73 out of 100, grade Call allowedShopify . Cloudflare
171godaddy.comScore 78 out of 100, grade B, partial assessmentall allowedAkamai
173amazon-adsystem.comrefused GPTBotScore 78 out of 100, grade B, partial assessmentall allowedunidentified
175paypal.comrefused GPTBotScore 92 out of 100, grade A, partial assessmentall allowedFastly
179firefox-portal-detection.comScore 78 out of 100, grade B, partial assessmentall allowedFastly
181flickr.comScore 42 out of 100, grade D, partial assessment6 blockedunidentified
182discord.comScore 77 out of 100, grade Ball allowedWebflow . Cloudflare
183forter.comScore 87 out of 100, grade B1 blockedWordPress . Cloudflare
192cnn.comScore 31 out of 100, grade F9 blockedFastly
196kaspersky.comScore 86 out of 100, grade Ball allowedNext.js
198ebay.comScore 60 out of 100, grade C, partial assessment5 blockedAkamai
200brave.comScore 75 out of 100, grade Ball allowedAmazon CloudFront
202soundcloud.comScore 31 out of 100, grade F, partial assessment10 blockedAmazon CloudFront
203theguardian.comScore 46 out of 100, grade D6 blockedunidentified
206unity3d.comScore 79 out of 100, grade Ball allowedSanity
209medium.comrefused GPTBotScore 46 out of 100, grade D4 blockedCloudflare
210hcaptcha.comScore 81 out of 100, grade Ball allowedCloudflare
212stripe.comScore 80 out of 100, grade Ball allowedNext.js
213roku.comScore 77 out of 100, grade Ball allowedNext.js . Cloudflare
214publicnode.comScore 59 out of 100, grade Dall allowedNext.js . Cloudflare
215oracle.comScore 78 out of 100, grade B, partial assessmentall allowedAkamai
219bbc.comScore 31 out of 100, grade F10 blockedNext.js . Fastly
222afternic.comScore 60 out of 100, grade Call allowedContentful . Akamai
226weather.comScore 82 out of 100, grade Ball allowedNext.js . Akamai
227forbes.comrefused GPTBotScore 57 out of 100, grade D, partial assessment5 blockedFastly
229namecheap.comrefused GPTBotScore 92 out of 100, grade A, partial assessmentall allowedCloudflare
231sciencedirect.comrefused GPTBotScore 78 out of 100, grade B, partial assessmentall allowedCloudflare
234xiaomi.comScore 60 out of 100, grade Call allowedAkamai
243cisco.comScore 92 out of 100, grade A, partial assessmentall allowedAkamai
245applovin.comScore 79 out of 100, grade Ball allowedNext.js . Google Cloud / GFE
249dnsmadeeasy.comScore 55 out of 100, grade Dall allowedAdobe Experience Manager . Fastly
254centinel-analytica.comScore 88 out of 100, grade Ball allowedNext.js . Cloudflare
257salesforce.comScore 77 out of 100, grade Ball allowedContentful . Akamai
263hubspot.comScore 82 out of 100, grade Ball allowedHubSpot CMS . Cloudflare
265imdb.comScore 9 out of 100, grade F, partial assessment11 blockedunidentified
266googleblog.comScore 57 out of 100, grade Dall allowedGoogle Cloud / GFE
267alidns.comScore 55 out of 100, grade Dall allowedAlibaba Cloud CDN
269arubanetworks.comScore 78 out of 100, grade B, partial assessmentall allowedAkamai
272icanhazip.comScore 84 out of 100, grade B, partial assessmentall allowedCloudflare
278launchdarkly.comScore 73 out of 100, grade Call allowedNext.js . Netlify
279slack.comScore 75 out of 100, grade Ball allowedunidentified
283ibm.comScore 73 out of 100, grade C, partial assessmentall allowedAdobe Experience Manager . Akamai
284hp.comScore 62 out of 100, grade Call allowedAdobe Experience Manager . Akamai
285reuters.comScore 26 out of 100, grade F, partial assessment9 blockedunidentified
291booking.comScore 92 out of 100, grade A, partial assessmentall allowedAmazon CloudFront
293withgoogle.comrefused GPTBotScore 42 out of 100, grade Dall allowedGoogle Cloud / GFE
294nature.comScore 45 out of 100, grade D, partial assessment7 blockedFastly
297steampowered.comScore 55 out of 100, grade Dall allowedunidentified
298canva.comrefused GPTBotScore 66 out of 100, grade C, partial assessment4 blockedCloudflare
299gamepass.comScore 52 out of 100, grade Dall allowedunidentified
304ip-api.comScore 60 out of 100, grade Call allowedunidentified
310grammarly.comScore 64 out of 100, grade Call allowedContentful . Amazon CloudFront
311springer.comScore 92 out of 100, grade A, partial assessmentall allowedFastly
316zendesk.comScore 86 out of 100, grade Ball allowedNext.js . Cloudflare
317miui.comScore 52 out of 100, grade Dall allowedNext.js
321wiley.comrefused GPTBotScore 47 out of 100, grade Dall allowedCloudflare
322anthropic.comScore 64 out of 100, grade Call allowedNext.js . Cloudflare
323avast.comScore 62 out of 100, grade Call allowedAdobe Experience Manager . Fastly
325vungle.comScore 76 out of 100, grade Ball allowedWordPress . Cloudflare
327alibaba.comScore 53 out of 100, grade D2 blockedAlibaba Cloud CDN
331weibo.comScore 18 out of 100, grade F, partial assessment11 blockedunidentified
340espn.comScore 72 out of 100, grade C, partial assessment3 blockedAmazon CloudFront
341trustpilot.comScore 66 out of 100, grade C, partial assessment2 blockedunidentified
343autodesk.comScore 78 out of 100, grade B, partial assessmentall allowedAkamai
345etsy.comrefused GPTBotScore 84 out of 100, grade B, partial assessmentall allowedFastly
346paloaltonetworks.comScore 71 out of 100, grade Call allowedAkamai
347unisys.comScore 83 out of 100, grade Ball allowedCloudflare
350smartadserver.comScore 55 out of 100, grade Dall allowedWordPress . Cloudflare
351naver.comScore 6 out of 100, grade F, partial assessment11 blockedunidentified
352okta.comScore 77 out of 100, grade Ball allowedAdobe Experience Manager . Cloudflare
353ea.comScore 66 out of 100, grade C1 blockedNext.js
355tp-link.comScore 74 out of 100, grade Call allowedAmazon CloudFront
356bilibili.comrefused GPTBotScore 57 out of 100, grade Dall allowedunidentified
357userapi.comScore 48 out of 100, grade Dall allowedunidentified
358nvidia.comScore 82 out of 100, grade Ball allowedAdobe Experience Manager
359tradingview.comScore 74 out of 100, grade Call allowedAmazon CloudFront
361dynatrace.comScore 89 out of 100, grade Ball allowedNuxt . Amazon CloudFront
363dell.comScore 80 out of 100, grade Ball allowedAdobe Experience Manager . Akamai
366dotomi.comScore 69 out of 100, grade Call allowedNext.js . Amazon CloudFront
367checkpoint.comrefused GPTBotScore 70 out of 100, grade Call allowedWordPress . Amazon CloudFront
369mi.comScore 60 out of 100, grade Call allowedAkamai
370elasticbeanstalk.comScore 67 out of 100, grade Call allowedAmazon CloudFront
371wsj.comScore 31 out of 100, grade F, partial assessment8 blockedunidentified
372weebly.comScore 69 out of 100, grade Call allowedCloudflare
376aliyun.comScore 55 out of 100, grade Dall allowedAlibaba Cloud CDN
378browser-intake-datadoghq.comrefused GPTBotScore 78 out of 100, grade B, partial assessmentall allowedunidentified
380netease.comScore 52 out of 100, grade Dall allowedunidentified
385gmail.comScore 64 out of 100, grade Call allowedGoogle Cloud / GFE
388xboxlive.comScore 55 out of 100, grade Dall allowedunidentified
392globo.comScore 56 out of 100, grade D2 blockedunidentified
393go.comScore 56 out of 100, grade D2 blockedAmazon CloudFront
394wps.comScore 81 out of 100, grade Ball allowedNuxt . Amazon CloudFront
397name.comScore 80 out of 100, grade B1 blockedSanity . Cloudflare
399gitlab.comScore 73 out of 100, grade Call allowedNuxt . Cloudflare
401amazontrust.comScore 52 out of 100, grade Dall allowedAmazon CloudFront
402issuu.comScore 64 out of 100, grade Call allowedNext.js . Fastly
404bloomberg.comrefused GPTBotScore 27 out of 100, grade F, partial assessment10 blockedFastly
407taobao.comScore 55 out of 100, grade Dall allowedAlibaba Cloud CDN
409opendns.comScore 58 out of 100, grade Dall allowedunidentified
418plesk.comScore 79 out of 100, grade Ball allowedWordPress . Cloudflare
420yandex.comScore 56 out of 100, grade Dall allowedunidentified
421visualstudio.comScore 71 out of 100, grade Call allowedWordPress
423calendly.comScore 82 out of 100, grade Ball allowedAstro . Cloudflare
424firetvcaptiveportal.comScore 78 out of 100, grade B, partial assessmentall allowedunidentified
426substack.comScore 78 out of 100, grade B, partial assessmentall allowedCloudflare
427duckduckgo.comScore 64 out of 100, grade Call allowedNext.js
428163.comrefused GPTBotScore 51 out of 100, grade Dall allowedAlibaba Cloud CDN
429temu.comScore 74 out of 100, grade Call allowedunidentified
430aliexpress.comScore 71 out of 100, grade C, partial assessment3 blockedAlibaba Cloud CDN
433aol.comScore 50 out of 100, grade D3 blockedNext.js . Amazon CloudFront
435steamcommunity.comScore 55 out of 100, grade Dall allowedunidentified
437indeed.comrefused GPTBotScore 78 out of 100, grade B, partial assessmentall allowedCloudflare
438appsflyer.comScore 95 out of 100, grade Aall allowedWordPress . Cloudflare
443expireddomains.comrefused GPTBotScore 84 out of 100, grade B, partial assessmentall allowedCloudflare
448dailymotion.comScore 57 out of 100, grade Dall allowedAmazon CloudFront
455mcafee.comScore 78 out of 100, grade B, partial assessmentall allowedAkamai
456intel.comScore 92 out of 100, grade A, partial assessmentall allowedAkamai
457mzstatic.comScore 70 out of 100, grade Call allowedunidentified
465duolingo.comScore 52 out of 100, grade Dall allowedAmazon CloudFront
467pixabay.comrefused GPTBotScore 67 out of 100, grade C, partial assessment4 blockedCloudflare
470bluehost.comScore 78 out of 100, grade B, partial assessmentall allowedCloudflare
471braze.comScore 73 out of 100, grade Call allowedSanity . Cloudflare
472trendmicro.comScore 77 out of 100, grade Ball allowedAdobe Experience Manager
477dropcatch.comScore 48 out of 100, grade Dall allowedCloudflare
478shein.comScore 71 out of 100, grade Call allowedCloudflare
479goodreads.comScore 56 out of 100, grade D1 blockedAmazon CloudFront
480teamviewer.comrefused GPTBotScore 64 out of 100, grade Call allowedNext.js . Cloudflare
481walmart.comScore 74 out of 100, grade Call allowedNext.js
483huawei.comScore 71 out of 100, grade Call allowedSitecore
486cnbc.comrefused GPTBotScore 24 out of 100, grade F11 blockedAkamai
490markmonitor.comrefused GPTBotScore 78 out of 100, grade B, partial assessmentall allowedCloudflare
492businessinsider.comScore 72 out of 100, grade C2 blockedFastly
493wp.comrefused GPTBotScore 77 out of 100, grade Ball allowedWordPress
497eventbrite.comScore 64 out of 100, grade Call allowedNext.js . Amazon CloudFront
499hostgator.comScore 78 out of 100, grade B, partial assessmentall allowedCloudflare
500scribd.comScore 78 out of 100, grade B, partial assessmentall allowedNext.js . Fastly
501tplinkcloud.comScore 52 out of 100, grade Dall allowedAmazon CloudFront
503foxnews.comScore 89 out of 100, grade Ball allowedFastly
504quora.comScore 13 out of 100, grade F, partial assessment10 blockedunidentified
505oraclecloud.comScore 78 out of 100, grade B, partial assessmentall allowedAkamai
506lijit.comScore 49 out of 100, grade Dall allowedAmazon CloudFront
509statista.comrefused GPTBotScore 43 out of 100, grade D7 blockedAmazon CloudFront
512playstation.comScore 73 out of 100, grade Call allowedAdobe Experience Manager . Akamai
514addtoany.comScore 55 out of 100, grade Dall allowedCloudflare

Showing the 200 most-visited.

Using these figures

Free to reuse in research, journalism or a product under CC BY 4.0, with credit to Fidget Labs BV. Quote the measurement date so the claim stays checkable as the index moves.

CrawlIndex by Fidget Labs BV. "Agent readiness on .com." https://crawlindex.org (measured 2026-11-22). Licensed CC BY 4.0.