The Weekly Crawl — every issue

Which domains changed their AI-crawler policy, which crawler, and when.

20 Sept 2026
1349 domains changed AI-crawler policy across 9706 crawler states, led by sciencenews.org.
13 Sept 2026
201 domains changed AI-crawler policy across 1067 crawler states, led by elle.fr.
06 Sept 2026
183 domains changed AI-crawler policy across 1207 crawler states, led by epochtimes.com.
30 Aug 2026
179 domains changed AI-crawler policy across 1136 crawler states, led by fastcompany.com.

4 issues. Figures are as published at the time and are not restated later; corrections appear in the changelog.
Get this by email, free · Method