Change feed
What moved recently
Robots.txt edits are never announced. This is every movement the crawler has detected, newest first. A domain's first measurement is a baseline and is not listed, and comparisons are never made across a probe change or a change of crawl location.
No changes recorded yet. The index needs at least two comparable crawls of a domain before it can report movement, so the first entries appear after the next run.
Using these figures
Free to reuse in research, journalism or a product under CC BY 4.0, with credit to Fidget Labs BV. Quote the measurement date so the claim stays checkable as the index moves.
CrawlIndex by Fidget Labs BV. "Changes in AI crawler policy." https://crawlindex.org (measured 2026-08-09). Licensed CC BY 4.0.