Everything You Want to Know about SimilarwebBot
The Similarweb Bot is the crawler behind Similarweb's Site Audit tool, crawling a site's pages to collect data for website analytics.
Last verified August 15, 2026 against Similarweb's published crawler docs
Why it's crawling your site
A Similarweb customer ran a Site Audit crawl against your domain to collect analytics data.
How SimilarwebBot views your website
SimilarwebBot fetches raw page HTML (full page html; can spoof googlebot-smartphone ua by default) — it does not execute JavaScript. Content that only appears after client-side rendering is invisible to it.
Paste a URL for the rendered-vs-crawled diff, blocked resources, and the sections that never reach the bot.
How to block SimilarwebBot
Add the rule below to your site's robots.txt. Most crawlers, including SimilarwebBot, check for changes within 24 hours.
User-agent: similarweb
Disallow: /Need the full walkthrough?
Partial blocks, per-directory rules, Cloudflare and server-level options, verification.