AI crawler directory
What is OAI-SearchBot?
Last verified
OpenAI's search crawler. It is used to surface websites in ChatGPT's search features. OpenAI says sites opted out of OAI-SearchBot are not shown in ChatGPT search answers, though they can still appear as navigational links. Robots.txt changes take about 24 hours to apply.
OAI-SearchBot at a glance
- Operator
- OpenAI
- Kind
- Crawler: fetches on its own schedule
- Used for
- Search index
- Feeds
- ChatGPT search
- robots.txt token
OAI-SearchBot- User agent string
Mozilla/5.0 (Macintosh; Intel Mac OS X 10_15_7) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/131.0.0.0 Safari/537.36; compatible; OAI-SearchBot/1.4; +https://openai.com/searchbot
- Follows robots.txt
- Yes. OpenAI documents the OAI-SearchBot robots.txt tag as the search control.
- Runs JavaScript
- Not documented by the operator
- Published IP ranges
- searchbot.json: https://openai.com/searchbot.jsonCheck an address against these ranges
- Official source
- Overview of OpenAI Crawlershttps://developers.openai.com/api/docs/bots
- Verified
Every AI crawler in one table
OAI-SearchBot is one of 29 crawlers and fetchers in the AI crawler directory, each checked against its operator's own documentation.