For publishers
Enter your site. We read your public robots.txt using the robots.txt standard (RFC 9309), then count how often eight AI engines cited your site in our monitoring data over the last 90 days. Free, no account needed.
Each host is counted on its own: uk.example.com is separate from example.com.
On November 21st, 2026 we read the robots.txt of the 100 sites AI engines cited most in our data. 92 of the 98 we could read let every AI search crawler in. Six block at least one and are still cited in our data: Forbes blocks PerplexityBot, and Perplexity cited Forbes 289 times. Our data can't show how an engine reached a page; a licensing deal, a search partner's index or a link a person shared could each explain it. Read the study.
robots.txt decides who may crawl. This lookup puts that next to who names you in answers, for your own site.
Want the raw counts for your site, or a look at specific topics? Email support@aeospotlight.com.