Everything You Want to Know about SemrushBot-ES
SemrushBot-ES is one of Semrush's specialized crawler variants used for its on-page and technical SEO analysis tools.
Last verified August 15, 2026 against Semrush's published crawler docs
Why it's crawling your site
Triggered by Semrush customers running site-audit style reports against your domain.
How SemrushBot-ES views your website
SemrushBot-ES fetches raw page HTML (full page html) — it does not execute JavaScript. Content that only appears after client-side rendering is invisible to it.
Paste a URL for the rendered-vs-crawled diff, blocked resources, and the sections that never reach the bot.
How to block SemrushBot-ES
Add the rule below to your site's robots.txt. Most crawlers, including SemrushBot-ES, check for changes within 24 hours.
User-agent: SemrushBot-ES
Disallow: /Need the full walkthrough?
Partial blocks, per-directory rules, Cloudflare and server-level options, verification.