Everything You Want to Know about GoogleBot
Googlebot is Google's primary web crawler, used to keep Google Search (including Discover, Images, Video, News) fresh and up to date.
Last verified August 15, 2026 against Google's published crawler docs
Why it's crawling your site
Googlebot crawls your site to discover and index content for Google Search results; it always respects robots.txt for automatic crawls.
How GoogleBot views your website
GoogleBot renders JavaScript before reading your page, so client-side content, hydrated components, and JS-injected text are all visible to it, the same as a browser would see.
Paste a URL for the rendered-vs-crawled diff, blocked resources, and the sections that never reach the bot.
How to block GoogleBot
Add the rule below to your site's robots.txt. Most crawlers, including GoogleBot, check for changes within 24 hours.
User-agent: Googlebot
Disallow: /Need the full walkthrough?
Partial blocks, per-directory rules, Cloudflare and server-level options, verification.