Can ChatGPT, Perplexity, Claude, Google and Bing reach your site? Enter your address. We read your robots.txt the way the crawlers do and show which AI crawlers it lets in, which it blocks, and the line that does it. Free, no account needed.
Enter a page address to also check that page's path.
User-agent: *, and the longest matching rule wins.We read the robots.txt the public sees. A few large sites show verified crawlers (for example, the real Googlebot) different rules. A firewall or bot protection service can still block crawlers that robots.txt allows; the site scan fetches your pages the way they do.
For AI search and citations: OAI-SearchBot (ChatGPT search), PerplexityBot, Claude-SearchBot, Googlebot (Google Search and AI Overviews) and Bingbot (Bing, which Microsoft Copilot draws on). GPTBot and ClaudeBot only collect training data.
That is a business choice. GPTBot collects training data for OpenAI models. Blocking it does not remove you from ChatGPT search, which uses OAI-SearchBot.
Under the robots.txt standard (RFC 9309), a missing file means every crawler may fetch every page.
No. Google documents Google-Extended as a training opt-out for Gemini and Vertex AI. Google Search and AI Overviews use Googlebot.
It is optional. It is a proposed plain-text summary of your site for language models. No major AI engine has confirmed that it reads one.
Not necessarily. Access is the first step: it lets AI engines fetch your pages. Whether they name you depends on what they find. Our free Brand Mention Check shows what ChatGPT and Perplexity say about your brand today.