What OpenAI says each crawler does
OpenAI’s crawler documentation lists four user agents. The three below are the ones that bear on ChatGPT search and model training.
The same page says “Each setting is independent of the others,” and gives the example of a site that allows OAI-SearchBot so it can appear in search results while disallowing GPTBot so its content stays out of training.
Five brands that block GPTBot and still get recommended
We read the robots.txt file of every domain in DecaGEO’s ChatGPT rankings for the week of October 4, 2026. That is 312 domains across 10 B2B software categories, and 283 of them returned a file we could read on October 8. A domain counts as blocking a crawler when the group that applies to it (its ownUser-agent group, or * when it has none) disallows the site root.
Seven of the 283 blocked GPTBot. Four blocked OAI-SearchBot, and one blocked Bingbot. Five of the seven GPTBot blockers ranked in the top 11 of their category that week.
Ranking week of October 4, 2026 · robots.txt read October 8, 2026, at the site root. Rankings: DecaGEO, ChatGPT (GPT-5.6 Terra), US English. Mention Rate is the share of the category’s ChatGPT answers that week that recommended the brand. Rank follows DECA Score, which also weights where in an answer a brand appears, so Midjourney ranks above Canva with a lower Mention Rate. Bingbot was allowed on all five domains. How the rankings are measured: methodology.
Midjourney and Shutterstock give GPTBot and OAI-SearchBot separate groups in the same file. GPTBot gets
Disallow: / and OAI-SearchBot is let in by name, which is the split OpenAI’s documentation describes. Figma names OAI-SearchBot in a group it shares with ChatGPT-User and other AI search agents, but that group opens only the pages it lists, such as the home and pricing pages, and ends with Disallow: / for the rest of the site.
Canva and Midjourney have both been recommended every week since July 26, 2026, the first week DecaGEO measured with GPT-5.6 Terra. Canva ranked #2 for nine straight weeks, then #3, with a Mention Rate between 51% and 62% each week. Midjourney ranked between #2 and #5, with a Mention Rate between 39% and 47%. The robots.txt files were read once, on October 8, 2026. Earlier versions of the files weren’t read.
Read the file on the domain that holds your product pages
Claude’s listed domain, claude.ai, blocks OAI-SearchBot along with GPTBot and ChatGPT-User. Claude still had a 27% Mention Rate among AI writing assistants that week. Anthropic’s other domains, claude.com (where Claude’s pricing page lives) and anthropic.com, allow every crawler. The rankings don’t show which of these domains ChatGPT read. A product often spans an app domain, a marketing domain and a docs subdomain, each with its own robots.txt. Check the file on the domain that serves your pricing and feature pages first, then the others.OAI-SearchBot blocks are rare among ranked brands
Of the four domains that blocked OAI-SearchBot at the root, one is claude.ai, whose sister domains are open. OpenAI states that sites opted out of OAI-SearchBot won’t be shown in ChatGPT search answers, though they can still appear as navigational links. DecaGEO’s GEO Guide checks a second line. Its Access step reads whether robots.txt lets both OAI-SearchBot and Bingbot in, because the Guide treats ChatGPT search as drawing on web results that Bingbot feeds it. OpenAI’s crawler page doesn’t mention Bing. One of the 283 domains blocked Bingbot at the root. If yours does, open the Bingbot group the same way you would open OAI-SearchBot.What to check, in order
1
Read your robots.txt
Open
https://<your-domain>/robots.txt on the domain that serves your product pages. Find the group that starts User-agent: OAI-SearchBot. If there isn’t one, the User-agent: * group applies. A Disallow line that covers your product pages keeps ChatGPT search away from them, unless an Allow line names those pages with a longer, more specific path. When both lines match a page, the one with the longer path wins. Read the Bingbot group the same way, and open it if it blocks them. OpenAI says search takes about 24 hours to pick up a robots.txt change.Whether to block GPTBot is a decision about model training. You can make it without touching your search visibility.2
Check what sits in front of your site
OpenAI recommends allowing requests from its published IP ranges as well as the robots.txt line. Bot protection at a CDN or firewall can turn the crawler away while the file says it’s allowed. With a free DecaGEO account, the GEO Guide’s Access step reads your robots.txt, reports Allowed or Blocked with the lines that decided it, and tells you when your site refuses the request instead.
3
Check the page itself
The GEO Guide’s Page step opens the one page you’d send a buyer the way a bot would. It checks that the page responds, allows indexing, and names itself as the main address. The Content step then shows what the bot reads on that page next to the buying question you gave it, so you can see whether the words match.
4
Watch whether you appear
The public ranking for your category shows who ChatGPT recommends each week, with no account. A free account tracks one brand and sends an alert through Add Your Brand when it enters the ranking. Pro tracks any number of brands with weekly change alerts.
FAQ
Should I unblock GPTBot to get recommended by ChatGPT?
The rankings give no reason to. Five brands that block GPTBot were recommended in the week of October 4, 2026, and OpenAI describes GPTBot as a training crawler that is independent of search. Decide on GPTBot by whether you want your content used for training.Does blocking OAI-SearchBot remove my brand from ChatGPT?
OpenAI says a site that opts out of OAI-SearchBot won’t be shown in ChatGPT search answers, though it can still appear as a navigational link.Which domain’s robots.txt counts?
Each host has its own file, andwww.figma.com and weave.figma.com are read separately. Start with the host that serves your pricing and product pages.