Everything You Want to Know about OAI-SearchBot
OAI-SearchBot crawls and indexes pages to surface and cite websites in ChatGPT's search features. It is not used for foundation-model training.
Last verified August 15, 2026 against OpenAI's published crawler docs
Why it's crawling your site
OpenAI crawls your site with OAI-SearchBot so your pages can appear in ChatGPT's search results and citations; blocking it removes your site from ChatGPT search answers, though navigational links may still show.
How OAI-SearchBot views your website
OAI-SearchBot fetches raw page HTML (full page html, no js rendering documented) — it does not execute JavaScript. Content that only appears after client-side rendering is invisible to it.
Paste a URL for the rendered-vs-crawled diff, blocked resources, and the sections that never reach the bot.
How to block OAI-SearchBot
Add the rule below to your site's robots.txt. Most crawlers, including OAI-SearchBot, check for changes within 24 hours.
User-agent: OAI-SearchBot
Disallow: /Need the full walkthrough?
Partial blocks, per-directory rules, Cloudflare and server-level options, verification.