Edge network
Cloudflare
1,132 measured sites. 114 of them, 10.1%, block at least one answer-surface AI crawler. Mean readiness score 70.4.
Compare against every other edge network
| Rank | Domain | Score | Answer-surface crawlers | Stack |
|---|---|---|---|---|
| 2 | cloudflare.com | Score 85 out of 100, grade B | all allowed | Astro . Cloudflare |
| 57 | cloudflare.netrefused GPTBot | Score 78 out of 100, grade B, partial assessment | all allowed | Cloudflare |
| 77 | zoom.us | Score 77 out of 100, grade B | all allowed | Cloudflare |
| 80 | workers.dev | Score 67 out of 100, grade C | all allowed | Astro . Cloudflare |
| 82 | openai.comrefused GPTBot | Score 57 out of 100, grade D | all allowed | Contentful . Cloudflare |
| 88 | vimeo.com | Score 56 out of 100, grade D | 3 blocked | Next.js . Cloudflare |
| 139 | epicgames.comrefused GPTBot | Score 78 out of 100, grade B, partial assessment | all allowed | Cloudflare |
| 143 | nih.govrefused GPTBot | Score 78 out of 100, grade B, partial assessment | all allowed | Cloudflare |
| 144 | discord.gg | Score 73 out of 100, grade C | all allowed | Webflow . Cloudflare |
| 168 | discord.com | Score 77 out of 100, grade B | all allowed | Webflow . Cloudflare |
| 169 | paypal.com | Score 80 out of 100, grade B | all allowed | Cloudflare |
| 178 | shopify.com | Score 80 out of 100, grade B | all allowed | Shopify . Cloudflare |
| 183 | base.org | Score 70 out of 100, grade C | all allowed | Next.js . Cloudflare |
| 184 | hosting24.comrefused GPTBot | Score 65 out of 100, grade C | all allowed | Nuxt . Cloudflare |
| 191 | w3.org | Score 84 out of 100, grade B, partial assessment | all allowed | Cloudflare |
| 198 | creativecommons.org | Score 73 out of 100, grade C | all allowed | WordPress . Cloudflare |
| 202 | medium.comrefused GPTBot | Score 46 out of 100, grade D | 4 blocked | Cloudflare |
| 203 | vungle.com | Score 76 out of 100, grade B | all allowed | HubSpot CMS . Cloudflare |
| 213 | doi.org | Score 54 out of 100, grade D | all allowed | Cloudflare |
| 220 | sciencedirect.comrefused GPTBot | Score 78 out of 100, grade B, partial assessment | all allowed | Cloudflare |
| 225 | researchgate.netrefused GPTBot | Score 84 out of 100, grade B, partial assessment | all allowed | Cloudflare |
| 230 | pages.dev | Score 70 out of 100, grade C | all allowed | Astro . Cloudflare |
| 237 | nist.gov | Score 67 out of 100, grade C | all allowed | Drupal . Cloudflare |
| 240 | cpanel.net | Score 77 out of 100, grade B | all allowed | WordPress . Cloudflare |
| 246 | canva.comrefused GPTBot | Score 66 out of 100, grade C, partial assessment | 4 blocked | Cloudflare |
| 248 | sourceforge.net | Score 92 out of 100, grade A, partial assessment | all allowed | Cloudflare |
| 252 | roku.com | Score 70 out of 100, grade C | all allowed | Cloudflare |
| 262 | hubspot.com | Score 82 out of 100, grade B | all allowed | HubSpot CMS . Cloudflare |
| 263 | hcaptcha.com | Score 78 out of 100, grade B | all allowed | Cloudflare |
| 269 | who.int | Score 76 out of 100, grade B | all allowed | Cloudflare |
| 271 | forter.com | Score 87 out of 100, grade B | 1 blocked | HubSpot CMS . Cloudflare |
| 285 | appsflyer.com | Score 78 out of 100, grade B | all allowed | WordPress . Cloudflare |
| 290 | namecheap.comrefused GPTBot | Score 92 out of 100, grade A, partial assessment | all allowed | Cloudflare |
| 322 | wiley.comrefused GPTBot | Score 57 out of 100, grade D | all allowed | Cloudflare |
| 334 | hostinger.com | Score 89 out of 100, grade B | all allowed | Nuxt . Cloudflare |
| 343 | smartadserver.com | Score 55 out of 100, grade D | all allowed | WordPress . Cloudflare |
| 346 | name.com | Score 83 out of 100, grade B | all allowed | Sanity . Cloudflare |
| 351 | doubleverify.com | Score 57 out of 100, grade D | all allowed | HubSpot CMS . Cloudflare |
| 366 | weebly.com | Score 69 out of 100, grade C | all allowed | Cloudflare |
| 368 | zendesk.com | Score 89 out of 100, grade B | all allowed | Next.js . Cloudflare |
| 373 | cookiedatabase.orgrefused GPTBot | Score 78 out of 100, grade B, partial assessment | all allowed | Cloudflare |
| 374 | hostgator.com | Score 78 out of 100, grade B, partial assessment | all allowed | Cloudflare |
| 376 | indeed.comrefused GPTBot | Score 84 out of 100, grade B, partial assessment | all allowed | Cloudflare |
| 383 | calendly.com | Score 79 out of 100, grade B | all allowed | Astro . Cloudflare |
| 384 | gitlab.comrefused GPTBot | Score 66 out of 100, grade C | all allowed | Nuxt . Cloudflare |
| 388 | plesk.com | Score 82 out of 100, grade B | all allowed | WordPress . Cloudflare |
| 407 | onelink.me | Score 75 out of 100, grade B | all allowed | WordPress . Cloudflare |
| 414 | expireddomains.comrefused GPTBot | Score 78 out of 100, grade B, partial assessment | all allowed | Cloudflare |
| 433 | temu.com | Score 78 out of 100, grade B, partial assessment | all allowed | Cloudflare |
| 436 | claude.airefused GPTBot | Score 48 out of 100, grade D | 4 blocked | Cloudflare |
| 444 | dropcatch.com | Score 48 out of 100, grade D | all allowed | Cloudflare |
| 447 | skyhigh.cloud | Score 73 out of 100, grade C | all allowed | WordPress . Cloudflare |
| 452 | substack.com | Score 78 out of 100, grade B, partial assessment | all allowed | Cloudflare |
| 464 | pixabay.comrefused GPTBot | Score 67 out of 100, grade C, partial assessment | 4 blocked | Cloudflare |
| 469 | speedtest.netrefused GPTBot | Score 84 out of 100, grade B, partial assessment | all allowed | Cloudflare |
| 491 | addtoany.com | Score 55 out of 100, grade D | all allowed | Cloudflare |
| 493 | ietf.orgrefused GPTBot | Score 53 out of 100, grade D | all allowed | Cloudflare |
| 497 | okta.com | Score 77 out of 100, grade B | all allowed | Adobe Experience Manager . Cloudflare |
| 502 | bluehost.com | Score 78 out of 100, grade B, partial assessment | all allowed | Cloudflare |
| 503 | mediatek.com | Score 64 out of 100, grade C | all allowed | HubSpot CMS . Cloudflare |
| 508 | markmonitor.comrefused GPTBot | Score 78 out of 100, grade B, partial assessment | all allowed | Cloudflare |
| 512 | stackoverflow.comrefused GPTBot | Score 78 out of 100, grade B, partial assessment | all allowed | Cloudflare |
| 532 | shop.apprefused GPTBot | Score 92 out of 100, grade A, partial assessment | all allowed | Cloudflare |
| 536 | ft.comrefused GPTBot | Score 53 out of 100, grade D, partial assessment | 6 blocked | Cloudflare |
| 539 | fandom.com | Score 78 out of 100, grade B, partial assessment | all allowed | Cloudflare |
| 540 | oup.comrefused GPTBot | Score 63 out of 100, grade C | all allowed | WordPress . Cloudflare |
| 551 | jquery.com | Score 63 out of 100, grade C | all allowed | WordPress . Cloudflare |
| 554 | wired.com | Score 51 out of 100, grade D | 8 blocked | Cloudflare |
| 555 | bitdefender.net | Score 57 out of 100, grade D | all allowed | Adobe Experience Manager . Cloudflare |
| 559 | tandfonline.comrefused GPTBot | Score 86 out of 100, grade B, partial assessment | 1 blocked | Cloudflare |
| 563 | digitalocean.com | Score 64 out of 100, grade C | all allowed | Next.js . Cloudflare |
| 584 | teamviewer.comrefused GPTBot | Score 67 out of 100, grade C | all allowed | Adobe Experience Manager . Cloudflare |
| 585 | cookielaw.orgrefused GPTBot | Score 63 out of 100, grade C | all allowed | Adobe Experience Manager . Cloudflare |
| 593 | dreamhost.com | Score 73 out of 100, grade C | all allowed | Next.js . Cloudflare |
| 595 | klaviyo.com | Score 86 out of 100, grade B | all allowed | HubSpot CMS . Cloudflare |
| 602 | braze.com | Score 73 out of 100, grade C | all allowed | Sanity . Cloudflare |
| 611 | berkeley.edu | Score 70 out of 100, grade C | all allowed | WordPress . Cloudflare |
| 614 | anthropic.com | Score 64 out of 100, grade C | all allowed | Sanity . Cloudflare |
| 624 | onetrust.com | Score 86 out of 100, grade B | all allowed | Adobe Experience Manager . Cloudflare |
| 630 | nike.comrefused GPTBot | Score 92 out of 100, grade A, partial assessment | all allowed | Cloudflare |
| 634 | doctolib.frrefused GPTBot | Score 86 out of 100, grade B, partial assessment | 1 blocked | Cloudflare |
| 635 | ikea.com | Score 71 out of 100, grade C | all allowed | Astro . Cloudflare |
| 638 | branch.io | Score 89 out of 100, grade B | all allowed | WordPress . Cloudflare |
| 645 | xhamster.com | Score 60 out of 100, grade C | all allowed | Cloudflare |
| 652 | loc.govrefused GPTBot | Score 78 out of 100, grade B, partial assessment | all allowed | Cloudflare |
| 659 | patreon.comrefused GPTBot | Score 90 out of 100, grade A, partial assessment | 1 blocked | Cloudflare |
| 660 | sagepub.com | Score 65 out of 100, grade C | 1 blocked | Cloudflare |
| 670 | britannica.comrefused GPTBot | Score 92 out of 100, grade A, partial assessment | all allowed | Cloudflare |
| 679 | anydesk.comrefused GPTBot | Score 78 out of 100, grade B, partial assessment | all allowed | Cloudflare |
| 683 | cambridge.orgrefused GPTBot | Score 86 out of 100, grade B, partial assessment | 1 blocked | Cloudflare |
| 684 | perplexity.airefused GPTBot | Score 78 out of 100, grade B, partial assessment | all allowed | Cloudflare |
| 695 | wpguardian.io | Score 70 out of 100, grade C | all allowed | WordPress . Cloudflare |
| 705 | kick.comrefused GPTBot | Score 92 out of 100, grade A, partial assessment | all allowed | Cloudflare |
| 711 | fontawesome.com | Score 48 out of 100, grade D | all allowed | Cloudflare |
| 715 | iso.orgrefused GPTBot | Score 78 out of 100, grade B, partial assessment | all allowed | Cloudflare |
| 719 | ring.com | Score 68 out of 100, grade C | all allowed | Shopify . Cloudflare |
| 722 | wpguardian.com | Score 70 out of 100, grade C | all allowed | WordPress . Cloudflare |
| 727 | uber.com | Score 61 out of 100, grade C | all allowed | Cloudflare |
| 729 | itch.iorefused GPTBot | Score 65 out of 100, grade C | all allowed | Cloudflare |
| 736 | stripchat.com | Score 64 out of 100, grade C | all allowed | Cloudflare |
| 737 | globalsign.com | Score 56 out of 100, grade D | 5 blocked | Cloudflare |
| 745 | bugsnag.com | Score 60 out of 100, grade C | all allowed | Astro . Cloudflare |
| 762 | buydomains.comrefused GPTBot | Score 78 out of 100, grade B, partial assessment | all allowed | Cloudflare |
| 763 | gumgum.com | Score 86 out of 100, grade B | all allowed | Cloudflare |
| 765 | curseforge.com | Score 60 out of 100, grade C | all allowed | Next.js . Cloudflare |
| 768 | hostgator.com.br | Score 73 out of 100, grade C | all allowed | Next.js . Cloudflare |
| 771 | mediafire.comrefused GPTBot | Score 84 out of 100, grade B, partial assessment | all allowed | Cloudflare |
| 778 | box.com | Score 71 out of 100, grade C | all allowed | Drupal . Cloudflare |
| 785 | singular.net | Score 89 out of 100, grade B | all allowed | HubSpot CMS . Cloudflare |
| 794 | allaboutcookies.org | Score 78 out of 100, grade B, partial assessment | all allowed | Cloudflare |
| 796 | fiverr.comrefused GPTBot | Score 92 out of 100, grade A, partial assessment | all allowed | Cloudflare |
| 803 | visma.com | Score 73 out of 100, grade C | all allowed | Cloudflare |
| 805 | apnews.comrefused GPTBot | Score 54 out of 100, grade D, partial assessment | 6 blocked | Cloudflare |
| 811 | people.com | Score 37 out of 100, grade F, partial assessment | 8 blocked | Cloudflare |
| 812 | moloco.com | Score 64 out of 100, grade C | all allowed | Cloudflare |
| 813 | networkadvertising.org | Score 70 out of 100, grade C | all allowed | WordPress . Cloudflare |
| 816 | prnewswire.com | Score 74 out of 100, grade C | all allowed | Adobe Experience Manager . Cloudflare |
| 821 | 33across.comrefused GPTBot | Score 78 out of 100, grade B, partial assessment | all allowed | Cloudflare |
| 824 | liftoff.io | Score 76 out of 100, grade B | all allowed | HubSpot CMS . Cloudflare |
| 825 | discordapp.com | Score 77 out of 100, grade B | all allowed | Webflow . Cloudflare |
| 829 | zoho.comrefused GPTBot | Score 57 out of 100, grade D | all allowed | Cloudflare |
| 830 | deepl.com | Score 77 out of 100, grade B | all allowed | Contentful . Cloudflare |
| 833 | unpkg.com | Score 60 out of 100, grade C | all allowed | Cloudflare |
| 852 | onlyfans.comrefused GPTBot | Score 84 out of 100, grade B, partial assessment | all allowed | Cloudflare |
| 854 | discogs.comrefused GPTBot | Score 78 out of 100, grade B, partial assessment | all allowed | Cloudflare |
| 856 | zoom.com | Score 77 out of 100, grade B | all allowed | Cloudflare |
| 863 | pexels.comrefused GPTBot | Score 61 out of 100, grade C, partial assessment | 5 blocked | Cloudflare |
| 869 | oxylabs.io | Score 73 out of 100, grade C | all allowed | Next.js . Cloudflare |
| 870 | chess.com | Score 71 out of 100, grade C | all allowed | Cloudflare |
| 875 | it.comrefused GPTBot | Score 78 out of 100, grade B, partial assessment | all allowed | Cloudflare |
| 876 | life360.comrefused GPTBot | Score 78 out of 100, grade B, partial assessment | all allowed | Cloudflare |
| 898 | readthedocs.iorefused GPTBot | Score 50 out of 100, grade D | all allowed | Cloudflare |
| 899 | agora.io | Score 68 out of 100, grade C | all allowed | HubSpot CMS . Cloudflare |
| 908 | worldbank.org | Score 82 out of 100, grade B | all allowed | Adobe Experience Manager . Cloudflare |
| 909 | fwmrm.net | Score 70 out of 100, grade C | all allowed | HubSpot CMS . Cloudflare |
| 910 | princeton.edu | Score 57 out of 100, grade D, partial assessment | all allowed | Drupal . Cloudflare |
| 912 | auvik.com | Score 80 out of 100, grade B | all allowed | WordPress . Cloudflare |
| 917 | wyzecam.com | Score 77 out of 100, grade B | all allowed | Shopify . Cloudflare |
| 929 | seedtag.com | Score 57 out of 100, grade D | all allowed | HubSpot CMS . Cloudflare |
| 938 | epam.comrefused GPTBot | Score 78 out of 100, grade B, partial assessment | all allowed | Cloudflare |
| 941 | flashtalking.com | Score 64 out of 100, grade C | all allowed | Cloudflare |
| 944 | genius.comrefused GPTBot | Score 78 out of 100, grade B, partial assessment | all allowed | Cloudflare |
| 954 | merriam-webster.comrefused GPTBot | Score 92 out of 100, grade A, partial assessment | all allowed | Cloudflare |
| 957 | thenai.org | Score 70 out of 100, grade C | all allowed | WordPress . Cloudflare |
| 959 | trendyol.comrefused GPTBot | Score 84 out of 100, grade B, partial assessment | all allowed | Cloudflare |
| 960 | character.airefused GPTBot | Score 92 out of 100, grade A, partial assessment | all allowed | Cloudflare |
| 962 | chaturbate.comrefused GPTBot | Score 92 out of 100, grade A, partial assessment | all allowed | Cloudflare |
| 966 | webmd.com | Score 51 out of 100, grade D | 2 blocked | Cloudflare |
| 973 | xhamster46.desi | Score 60 out of 100, grade C | all allowed | Cloudflare |
| 974 | ancestry.comrefused GPTBot | Score 78 out of 100, grade B, partial assessment | all allowed | Cloudflare |
| 979 | nexusmods.comrefused GPTBot | Score 78 out of 100, grade B, partial assessment | all allowed | Cloudflare |
| 982 | investopedia.com | Score 37 out of 100, grade F, partial assessment | 8 blocked | Cloudflare |
| 992 | conviva.com | Score 82 out of 100, grade B | all allowed | WordPress . Cloudflare |
| 1000 | pixiv.netrefused GPTBot | Score 52 out of 100, grade D, partial assessment | 5 blocked | Cloudflare |
| 1003 | faphouse.com | Score 58 out of 100, grade D | all allowed | Cloudflare |
| 1004 | flightradar24.comrefused GPTBot | Score 85 out of 100, grade B, partial assessment | 1 blocked | Cloudflare |
| 1014 | onesignal.com | Score 85 out of 100, grade B | all allowed | Cloudflare |
| 1018 | statcounter.comrefused GPTBot | Score 52 out of 100, grade D | all allowed | Cloudflare |
| 1019 | ox.ac.ukrefused GPTBot | Score 78 out of 100, grade B, partial assessment | all allowed | Cloudflare |
| 1024 | poki.com | Score 67 out of 100, grade C | all allowed | Cloudflare |
| 1025 | spamhaus.org | Score 60 out of 100, grade C | all allowed | Nuxt . Cloudflare |
| 1027 | ecosia.orgrefused GPTBot | Score 84 out of 100, grade B, partial assessment | all allowed | Cloudflare |
| 1031 | cloudinary.com | Score 89 out of 100, grade B | all allowed | WordPress . Cloudflare |
| 1034 | themeforest.net | Score 70 out of 100, grade C | all allowed | Cloudflare |
| 1035 | theatlantic.com | Score 50 out of 100, grade D | 8 blocked | Next.js . Cloudflare |
| 1038 | bidmachine.iorefused GPTBot | Score 52 out of 100, grade D | all allowed | Webflow . Cloudflare |
| 1046 | archiveofourown.orgrefused GPTBot | Score 47 out of 100, grade D | 2 blocked | Cloudflare |
| 1050 | columbia.edurefused GPTBot | Score 78 out of 100, grade B, partial assessment | all allowed | Cloudflare |
| 1051 | olx.com.brrefused GPTBot | Score 78 out of 100, grade B, partial assessment | all allowed | Cloudflare |
| 1052 | garmin.com | Score 55 out of 100, grade D, partial assessment | 1 blocked | Cloudflare |
| 1054 | letterboxd.comrefused GPTBot | Score 52 out of 100, grade D, partial assessment | 5 blocked | Cloudflare |
| 1055 | umich.edurefused GPTBot | Score 84 out of 100, grade B, partial assessment | all allowed | Cloudflare |
| 1058 | epicgames.devrefused GPTBot | Score 78 out of 100, grade B, partial assessment | all allowed | Cloudflare |
| 1064 | wise.com | Score 71 out of 100, grade C | all allowed | Next.js . Cloudflare |
| 1067 | maricopa.govrefused GPTBot | Score 54 out of 100, grade D | all allowed | Cloudflare |
| 1072 | alphonso.tv | Score 64 out of 100, grade C | all allowed | WordPress . Cloudflare |
| 1073 | netgear.com | Score 81 out of 100, grade B | all allowed | Cloudflare |
| 1074 | skroutz.grrefused GPTBot | Score 54 out of 100, grade D, partial assessment | 6 blocked | Cloudflare |
| 1081 | xhamster.desi | Score 60 out of 100, grade C | all allowed | Cloudflare |
| 1085 | hugedomains.comrefused GPTBot | Score 84 out of 100, grade B, partial assessment | all allowed | Cloudflare |
| 1089 | ilovepdf.com | Score 60 out of 100, grade C | all allowed | Cloudflare |
| 1091 | crazygames.com | Score 65 out of 100, grade C | all allowed | Next.js . Cloudflare |
| 1097 | oecd.orgrefused GPTBot | Score 92 out of 100, grade A, partial assessment | all allowed | Cloudflare |
| 1102 | tabelog.com | Score 56 out of 100, grade D | 2 blocked | Cloudflare |
| 1125 | ultimate-guitar.com | Score 52 out of 100, grade D | all allowed | Cloudflare |
| 1129 | kickstarter.comrefused GPTBot | Score 78 out of 100, grade B, partial assessment | all allowed | Cloudflare |
| 1131 | nsone.net | Score 78 out of 100, grade B, partial assessment | all allowed | Cloudflare |
| 1132 | usercentrics.eurefused GPTBot | Score 84 out of 100, grade B | all allowed | WordPress . Cloudflare |
| 1133 | hltv.orgrefused GPTBot | Score 78 out of 100, grade B, partial assessment | all allowed | Cloudflare |
| 1137 | xhaccess.com | Score 60 out of 100, grade C | all allowed | Cloudflare |
| 1143 | jimdo.com | Score 80 out of 100, grade B | all allowed | Astro . Cloudflare |
| 1144 | icanhazip.com | Score 84 out of 100, grade B, partial assessment | all allowed | Cloudflare |
| 1156 | typeform.com | Score 74 out of 100, grade C | all allowed | Webflow . Cloudflare |
| 1157 | clarin.com | Score 79 out of 100, grade B | all allowed | Next.js . Cloudflare |
| 1159 | rfc-editor.org | Score 64 out of 100, grade C | all allowed | Nuxt . Cloudflare |
| 1171 | pimpbunny.comrefused GPTBot | Score 32 out of 100, grade F | 8 blocked | Cloudflare |
| 1174 | auth0.com | Score 88 out of 100, grade B | all allowed | Next.js . Cloudflare |
| 1178 | plos.org | Score 80 out of 100, grade B | all allowed | HubSpot CMS . Cloudflare |
| 1194 | sahibinden.comrefused GPTBot | Score 78 out of 100, grade B, partial assessment | all allowed | Cloudflare |
| 1198 | rule34.xxxrefused GPTBot | Score 84 out of 100, grade B, partial assessment | all allowed | Cloudflare |
Showing the 200 most-visited.
Using these figures
Free to reuse in research, journalism or a product under CC BY 4.0, with credit to Fidget Labs BV. Quote the measurement date so the claim stays checkable as the index moves.
CrawlIndex by Fidget Labs BV. "Agent readiness on Cloudflare." https://crawlindex.org (measured 2026-09-22). Licensed CC BY 4.0.