Everything You Want to Know about SemrushBot-FT
SemrushBot-FT powers Semrush's Plagiarism Checker and related content-matching tools by verifying whether checked content appears on a given site.
Last verified August 15, 2026 against Semrush's published crawler docs
Why it's crawling your site
Triggered when a Semrush customer runs the Plagiarism Checker or a similar content-matching tool against your domain.
How SemrushBot-FT views your website
SemrushBot-FT fetches raw page HTML (full page html) — it does not execute JavaScript. Content that only appears after client-side rendering is invisible to it.
Paste a URL for the rendered-vs-crawled diff, blocked resources, and the sections that never reach the bot.
How to block SemrushBot-FT
Add the rule below to your site's robots.txt. Most crawlers, including SemrushBot-FT, check for changes within 24 hours.
User-agent: SemrushBot-FT
Disallow: /Need the full walkthrough?
Partial blocks, per-directory rules, Cloudflare and server-level options, verification.