Free tool
Is my site blocked from Gemini?
Last updated
The Gemini apps ground their answers in content from the Google Search index, which Googlebot crawls. Whether Google may use a site's content for that grounding, and for training future Gemini models, is decided by Google-Extended: a robots.txt token with no user agent of its own. Google says Google-Extended does not affect a site's inclusion in Google Search, so it has no effect on AI Overviews or AI Mode; it decides grounding in the Gemini apps and in Grounding with Google Search on Vertex AI.
Source: Google, Google's common crawlers (Google-Extended), read .
Eight requests: robots.txt, the homepage as a browser, and the homepage as six AI crawlers. Nothing is stored.
The crawlers that decide it for Gemini
- Google-ExtendedGoogle
- A robots.txt token. Decides whether content Google crawls may be used to train Gemini models and to ground answers in the Gemini apps. (Google, Google's common crawlers (Google-Extended), read )In this check: robots.txt verdict only. Google-Extended is a token, not a crawler: Google says it has no user agent of its own, so there is no request to probe.
- GooglebotGoogle
- Crawls for the Google Search index, which is what Gemini grounding draws on. (Google, Google's common crawlers (Google-Extended), read )In this check: robots.txt verdict only. CDNs such as Cloudflare verify Googlebot by its IP address, so a request that imitates it from our address is refused by design and would prove nothing.
Other ways a site is invisible to Gemini
Crawler access is the first condition of AI visibility, not the only one. This check does not measure the following.
- Not in Google's index. Grounding provides content from the Google Search index to the model, so a page that is not indexed is not there to be used. (Google, Google's common crawlers (Google-Extended))
- JavaScript is less of a barrier here than for other engines: Gemini uses Googlebot's infrastructure, which renders JavaScript, while the other AI crawlers measured did not run it. (Vercel and MERJ, The rise of the AI crawler (December 2024))
The free audit crawls up to 25 pages the way an answer engine reads them and reports pages whose content only appears with JavaScript, pages marked noindex and blocked crawlers, with the evidence. Whether a page is in a search index is reported by the index's owner, in Google Search Console or Bing Webmaster Tools; neither this check nor the audit sees it.