CommerceGeekBot: what it collects and how to opt out
CommerceGeekBot builds CommerceGeek's index of ecommerce sites: which platform and tools a store uses, and which features its product and listing pages have.
How it identifies itself
It sends this user agent: CommerceGeekBot/1.0 (+https://commercegeek.com/bot)
What it does
- Visits a small, hand-picked list of stores, about once a month.
- Makes at most 7 requests per store per visit (robots.txt, the sitemap, one listing page, one product page and, when they link to one, the store's returns page), at least 5 seconds apart.
- Obeys robots.txt, including
Crawl-delay. - Never visits the cart, checkout, account or search pages, and never adds anything to a basket.
- Stops if it meets bot protection, a login or a CAPTCHA, and doesn't come back for at least 25 days. It never tries to get past them.
What it keeps
Facts about technology and page features (for example "uses Klaviyo" or "product pages show a size guide"), each with a short piece of evidence such as the script address it came from. It doesn't keep page content, product data, prices or images, or anything about people.
How to opt out
Add this to your robots.txt and CommerceGeekBot will stay away:
User-agent: CommerceGeekBot
Disallow: /Or use this form. It takes effect straight away: the site is removed from the index and never visited again.