AI crawler directory
What is YouBot?
Last verified
The crawler that powers the You.com search engine, discovering and indexing pages for its results. You.com says it caches robots.txt for 30 minutes and signs its requests with Web Bot Auth.
YouBot at a glance
- Operator
- You.com
- Kind
- Crawler: fetches on its own schedule
- Used for
- Search index
- Feeds
- You.com search
- robots.txt token
YouBot- User agent string
Mozilla/5.0 AppleWebKit/537.36 (KHTML, like Gecko; compatible; YouBot/1.0; +https://docs.you.com/youbot; env:prod) Chrome/X.X.X.X Safari/537.36
- Follows robots.txt
- Yes. You.com: respects robots.txt, including user-agent rules and crawl-delay.
- Runs JavaScript
- Not documented by the operator
- Published IP ranges
- 68.67.112.0/24, stated on the documentation pageCheck an address against these ranges
- Official source
- YouBot: You.com's Web Crawlerhttps://docs.you.com/youbot
- Verified
Every AI crawler in one table
YouBot is one of 29 crawlers and fetchers in the AI crawler directory, each checked against its operator's own documentation.