BotForYou Updated 9 October 2026 · Common Crawl CC-MAIN-2026-39

Every AI bot · Google

googleother

A name associated with Google whose purpose hasn't been clearly documented.

In robots.txt: User-agent: googleother

The numbers

Named by1.4%of readable robots.txt files on the top 1M (8,410 of 600,979)
Of those, block it38.3%shut it out of the whole site (3,224 of 8,410)
Same sites, since CC-MAIN-2026-34+0.0 ptsshare of the same 394,690 sites that block it by name: 0.4% → 0.4%
Via Cloudflare's file1.9%of its blocks come from Cloudflare's ready-made robots.txt (62 of 3,224)
Block the whole site: 3,224Block some paths: 3,960Name it but allow it: 1,226
Every robots.txt file on the top 1M that names googleother, by what the rules that apply to it say.

Google's other bots

Sites often treat one company's bots differently, depending on what each is for.

BotPurposeFiles naming itOf those, block it
Google-ExtendedCollects pages to train AI62,05072.3%

Who does what

The highest-ranked sites on the top million in each group. Click one to see its whole robots.txt verdict.

Block some paths

  1. reuters.com #282
  2. tradingview.com #364
  3. wsj.com #373
  4. squarespace.com #544
  5. ouest-france.fr #1,071

Name it and allow it

  1. dailymail.co.uk #685
  2. gumgum.com #761
  3. adform.net #806
  4. waze.com #1,316
  5. akamai.com #1,889

Lists skip sites we don't showcase (adult content).

What this means for you: its purpose isn't documented, so what blocking it changes is unclear. Decide each bot on what it's for, not who runs it.

Counts use one robots.txt per site on this month's Tranco top 1M (600,979 files we could read). Bot list: registry 2026-07-14.1, kept in sync with Dark Visitors and ai.robots.txt.