AI crawler directory
What is Claude-User?
Last verified
Fetches pages when a person asks Claude a question. Anthropic says disabling it stops Claude retrieving the site's content in response to a user query, which may reduce the site's visibility for user-directed web search.
Claude-User at a glance
- Operator
- Anthropic
- Kind
- Fetcher: requests a page when a user asks
- Used for
- User-triggered fetch
- Feeds
- Claude
- robots.txt token
Claude-User- User agent string
- Anthropic names the robots.txt token but does not publish a full user agent string.
- Follows robots.txt
- Yes. Anthropic says its bots, including this one, honour robots.txt directives.
- Runs JavaScript
- Not documented by the operator
- Published IP ranges
- One list for all of Anthropic's bots: https://claude.com/crawling/bots.jsonCheck an address against these ranges
- Official source
- Does Anthropic crawl data from the web, and how can site owners block the crawler?https://support.claude.com/en/articles/8896518-does-anthropic-crawl-data-from-the-web-and-how-can-site-owners-block-the-crawler
- Verified
Every AI crawler in one table
Claude-User is one of 29 crawlers and fetchers in the AI crawler directory, each checked against its operator's own documentation.