Definition: what is OAI-SearchBot?
OAI-SearchBot is the crawler OpenAI uses to build the ChatGPT Search index. OpenAI operates three documented user agents with clearly separated jobs:
- OAI-SearchBot: crawls for the ChatGPT Search index — the basis for citations and links in answers with web search.
- GPTBot: collects training data for future models.
- ChatGPT-User: fetches pages live when a user triggers a specific URL or research task in ChatGPT.
The user-agent string contains OAI-SearchBot plus a verification URL; OpenAI publishes the corresponding IP ranges for authenticity checks. According to OpenAI's documentation, OAI-SearchBot and GPTBot follow robots.txt; for OAI-SearchBot, changes take effect after about 24 hours. For the user-initiated ChatGPT-User, OpenAI notes that robots.txt rules may not apply. OpenAI also documents OAI-AdsBot, which only checks landing pages submitted as ads.
The visibility consequence
Blocking OAI-SearchBot removes you from the source pool of ChatGPT Search — with more than 900 million weekly ChatGPT users according to OpenAI (as of February 2026), a consequential decision. What matters is separating the decisions: a brand can block training (GPTBot) and allow search (OAI-SearchBot) — or vice versa. Blanket blocking of “all AI crawlers” conflates three different business questions.
# allow search, block training User-agent: OAI-SearchBot Allow: / User-agent: GPTBot Disallow: /
Practice checks
- Check robots.txt: explicit rules per crawler instead of wildcard assumptions.
- WAF and bot protection: many blocks happen not in robots.txt but in firewalls and CDN rules — log analysis and status-code tests (200 instead of 403/429) belong in the audit.
- Verification: distinguish real OAI-SearchBot hits from fake crawlers via the published IP ranges.
- Monitoring: watch crawl frequency in the logs — it shows whether and how deeply the index covers the domain.
Related terms
OpenAI's crawler trio stands beside PerplexityBot, ClaudeBot and Google-Extended. Crawler policy is part of the retrieval layer of GEO; curated content hints are added by llms.txt. Full analysis: Technical SEO for AI crawlers.
Three OpenAI crawlers, three separate decisions
OAI-SearchBot (search), GPTBot (training) and ChatGPT-User (live fetch) answer different business questions. Blanket blocking decides your ChatGPT visibility without you noticing.
FAQ on OAI-SearchBot
What is the difference between OAI-SearchBot and GPTBot? ▾
OAI-SearchBot crawls for the ChatGPT Search index — it determines whether a domain can be cited and linked in answers. GPTBot collects training data for future models. Both can be controlled separately in robots.txt.
Should you block OAI-SearchBot? ▾
For most business models, no: blocking removes the domain from ChatGPT Search's source pool and costs citation visibility. It can make sense for paid-content models whose value an answer would pre-empt — that is a business decision, not an SEO reflex.
Does OAI-SearchBot respect robots.txt? ▾
According to OpenAI's documentation, yes, as does GPTBot; OpenAI applies robots.txt changes for OAI-SearchBot after about 24 hours. For the user-initiated ChatGPT-User, OpenAI notes the rules may not apply. OpenAI additionally publishes the crawlers' IP ranges so real hits can be verified and fake crawlers separated.
How do I check whether ChatGPT can reach my pages? ▾
In three steps: check robots.txt for explicit rules, analyse server logs for OAI-SearchBot hits with status 200 (403/429 indicate WAF blocks), and spot-test answers in ChatGPT Search for your own citations.