What is Google-Extended?
Google-Extended is not a separate crawler but a robots.txt token that lets publishers control whether content Google crawls may be used to train future Gemini models and for grounding in Gemini Apps and Vertex AI. According to Google, it does not affect inclusion in Google Search.
According to Google, Google-Extended has no separate HTTP user agent. Crawling happens with the existing Google user agents, and the token only works as a control. Searching your server logs for Google-Extended therefore finds nothing, and that is expected.
Its effect is narrower than many assume. Google states explicitly that Google-Extended neither affects a site’s inclusion in Google Search nor serves as a ranking signal. AI Overviews in Google Search rely on the regular search index that Googlebot builds. Blocking Google-Extended does not remove a page from Google Search.
What the token governs is use for Gemini models and for grounding in Gemini Apps and Grounding with Google Search on Vertex AI. If you do not want to appear there, block Google-Extended. If you want to stay visible in Google Search, do not block Googlebot.
What it means for your website
Decide separately: Googlebot for search, Google-Extended for Gemini training and grounding. The deeploupe GPTBot check reads the robots.txt rule for Google-Extended. There is no live test for it, and there cannot be one, because no separate user agent exists.
More on deeploupe
Sources
Terms help you understand. Whether AI crawlers can reach your site is something you measure.
Check your website for freefree · no signup · no credit card