BotForYou Updated 9 October 2026 · Common Crawl CC-MAIN-2026-39

Platforms

Shopify

24,891 sites on this month's Tranco top 1M whose robots.txt looks like Shopify's, compared with all 453,286 sites Common Crawl saved.

How sure: of the 24,553 Shopify guesses a server header or generator tag could check, 88.4% were right. A platform here is an informed guess, not a fact.

Shopify sites against every site

Block GPTBot by name1.5%
every site10.1%
List a sitemap98.3%
every site66.7%
Name any AI crawler7.3%
every site20.1%
Use Cloudflare's ready-made file0.9%
every site5.9%
Tell every crawler to stay out0.1%
every site2.7%

93.8% of live Shopify sites in our llms.txt sample publish one (1,785 of 1,902), against 13.7% of all live sampled sites.

Cloudflare stopped adding its ready-made file on 16 September 2026, so this month's Cloudflare row mixes files from before and after (what happened).

The biggest Shopify sites we found

  1. shop.app #547
  2. wyze.com #6,425
  3. boat-lifestyle.com #9,248
  4. nishatlinen.com #9,809
  5. jbhifi.com.au #9,914

Highest-ranked sites whose Shopify guess a server header or generator tag confirmed, skipping sites we don't showcase. Compare them side by side →