Tranco rank 209
t-online.de
t-online.de scores 56 out of 100 for agent readiness. It blocks 6 of 11 answer-surface AI crawlers.
Score 56 out of 100, grade D
- Last measured
- 2026-08-09 18:20 UTC
- First seen
- 2026-08-09
- Rubric / probe
- v1.0.0 / v2.0.0
Built on Next.js, served through Amazon CloudFront. Compare against everything else on the same stack.
How this score is made up
Agent access27.3 / 45
- Answer-surface crawlers allowed. 6 of 11 blocked: GPTBot, ClaudeBot, PerplexityBot, Google-Extended, Applebot-Extended, meta-externalagent.
- Secondary crawlers allowed. 2 of 12 blocked: CCBot, Bytespider.
- Serves crawlers the same content. A crawler and a browser receive comparable responses.
Machine-readable surface3 / 25
- robots.txt published. robots.txt is present and parseable.
- Sitemap declared in robots.txt. robots.txt does not declare a sitemap.
- llms.txt published. No /llms.txt.
- agents.md published. No /agents.md.
Content structure26 / 30
- Organization schema. Organization JSON-LD lets an agent resolve who publishes this site.
- WebSite schema. No WebSite JSON-LD.
- Additional structured data. Also declares: WebPage.
- Readable without JavaScript. 24,586 characters of text in the server response. Most crawlers do not execute JavaScript.
- Single top-level heading. 1 h1 element found.
- Semantic landmarks. Uses main, nav, header, footer, article, aside.
Crawler policy
Read from t-online.de/robots.txt. A crawler is listed as blocked when the rules deny it the site root. This operator names 8 AI crawlers explicitly, which means the policy is deliberate rather than inherited from a wildcard rule.
| Crawler | Operator | Tier | Named | Status |
|---|---|---|---|---|
| GPTBot | OpenAI | 1 | yes | Blocked |
| OAI-SearchBot | OpenAI | 1 | no | Allowed |
| ChatGPT-User | OpenAI | 1 | no | Allowed |
| ClaudeBot | Anthropic | 1 | yes | Blocked |
| Claude-User | Anthropic | 1 | no | Allowed |
| Claude-SearchBot | Anthropic | 1 | no | Allowed |
| PerplexityBot | Perplexity | 1 | yes | Blocked |
| Perplexity-User | Perplexity | 1 | no | Allowed |
| Google-Extended | 1 | yes | Blocked | |
| Applebot-Extended | Apple | 1 | yes | Blocked |
| meta-externalagent | Meta | 1 | yes | Blocked |
| CCBot | Common Crawl | 2 | yes | Blocked |
| Amazonbot | Amazon | 2 | no | Allowed |
| Bytespider | ByteDance | 2 | yes | Blocked |
| cohere-ai | Cohere | 2 | no | Allowed |
| MistralAI-User | Mistral | 2 | no | Allowed |
| YouBot | You.com | 2 | no | Allowed |
| DuckAssistBot | DuckDuckGo | 2 | no | Allowed |
| kagi-fetcher | Kagi | 2 | no | Allowed |
| Diffbot | Diffbot | 2 | no | Allowed |
| Google-NotebookLM | 2 | no | Allowed | |
| TavilyBot | Tavily | 2 | no | Allowed |
| FirecrawlAgent | Firecrawl | 2 | no | Allowed |
Show this score
Free to embed. Always reflects the latest measurement, and links back here so anyone can check the working.
<a href="https://crawlindex.org/site/t-online.de"><img src="https://crawlindex.org/badge/t-online.de.svg" alt="CrawlIndex agent readiness score for t-online.de" width="196" height="28"></a>Using these figures
Free to reuse in research, journalism or a product under CC BY 4.0, with credit to Fidget Labs BV. Quote the measurement date so the claim stays checkable as the index moves.
CrawlIndex by Fidget Labs BV. "t-online.de agent readiness." https://crawlindex.org (measured 2026-08-09). Licensed CC BY 4.0.