Why can't ChatGPT see my website?

ChatGPT can see websites. OpenAI sends crawlers to fetch pages for ChatGPT's search. If it can't see yours, something on your site is usually stopping it: a firewall setting, content that only appears after JavaScript runs, a login, or pages that never state what you do.

In our check of 59 websites' robots.txt files on 26 September 2026 (method below), none blocked ChatGPT. So the file most people worry about is rarely the problem.

Work through the five checks below, in order.

Can ChatGPT actually read websites?

Yes, ChatGPT can read websites. OpenAI's documentation says its crawler OAI-SearchBot "is used to surface websites in search results in ChatGPT's search features"¹. A second fetcher, ChatGPT-User, visits pages when a person's question needs them.

If you ask ChatGPT directly, it may tell you otherwise. When we asked "Why can't ChatGPT see my website?", it answered from memory, without searching, that it usually "can't 'see' your website live". That describes ChatGPT with search switched off, not how ChatGPT search finds businesses.

What are the five checks to find out why ChatGPT can't see my website?

1. Does your robots.txt block ChatGPT?

Your robots.txt file can block ChatGPT, but it rarely does. Open yoursite.com/robots.txt in your browser. Look for OAI-SearchBot, ChatGPT-User or GPTBot, and for a User-agent: * section followed by Disallow: /, which blocks every crawler.

Of the 59 robots.txt files we read, 0 blocked any of ChatGPT's crawlers. Only one blocked other AI crawlers (Claude, Perplexity and others): a US travel magazine, not a Malaysian business. Eight files mentioned GPTBot by name and still allowed it.

2. Is Cloudflare or a firewall blocking AI bots?

A Cloudflare or firewall setting can block ChatGPT without anything showing on your site, which makes it the check most owners miss. If your website uses Cloudflare, it has settings that block AI crawlers before they ever reach your robots.txt.

Cloudflare's own changelog says that from 15 September 2026, new domains joining Cloudflare get new defaults: "Bots classified as Training or as Agent are blocked on pages that display ads, while Search remains allowed." Cloudflare defines Agent as bots acting "in real time on a person's behalf, such as chat fetch bots"². ChatGPT-User fetching a page for someone's question is that kind of bot.

To check, open your domain in the Cloudflare dashboard and go to AI Crawl Control. Each crawler shows as allowed or blocked. The AI bot policies are under Security → Settings. Other firewalls and security plugins can have similar bot-blocking options, so ask whoever manages your site.

3. Can ChatGPT see your key facts, or are they only loaded by JavaScript?

In a December 2024 analysis of traffic on its network, Vercel found that the major AI crawlers, including OpenAI's and Anthropic's, did not run JavaScript³. If your prices, services or opening hours only appear after JavaScript runs, ChatGPT may see an almost empty page.

To check, open your page, press Cmd + Option + U on a Mac (Ctrl + U on Windows) to view the source, and search for a fact you care about, such as your price or your town. If it isn't in the source, a crawler that doesn't run JavaScript won't see it.

4. Is your website content hidden from ChatGPT behind a login, pop-up or PDF?

ChatGPT's crawlers can't log in or fill in forms. Content behind a members' area, a "request a quote" wall or a booking form is invisible to them. Put the essentials (what you do, where, rough prices) on an ordinary public page.

5. Does your website state the facts ChatGPT needs to recommend you?

A page ChatGPT can read can still have nothing worth citing. If your homepage says "quality service since 2008" but never names your services, areas or prices, ChatGPT has nothing to quote. We've shown which pages ChatGPT reads when customers ask in Malay or Chinese: mostly English pages with specific facts.

Does blocking GPTBot stop ChatGPT from recommending my business?

No, blocking GPTBot doesn't stop ChatGPT from recommending your business: GPTBot and ChatGPT search use different crawlers. OpenAI describes GPTBot as crawling content "that may be used in training our generative AI foundation models", and OAI-SearchBot as the one that surfaces sites in ChatGPT search¹. You can block GPTBot to opt out of training and still allow OAI-SearchBot, so ChatGPT search can find you.

How did we check which websites block ChatGPT?

On 26 September 2026 we fetched the robots.txt file of 167 websites: 126 that ChatGPT cited when we asked it Malaysian buying questions on 24 September, and 41 Malaysian business websites. We read each file the way crawlers do: a section naming the crawler overrides the general * section. A crawler counted as blocked only if the whole site was disallowed.

Only 62 of the 167 sites were reachable from our test network. Many large sites timed out from our connection. The findings above cover the 59 robots.txt files we could read, not all 167, and the sites we reached skew towards those on Cloudflare. Every site's result, including the unreachable ones, is in the downloadable CSV.

What can't this check tell me about my own website?

• Firewall blocks are invisible from outside. A Cloudflare setting can block real AI crawlers while the site looks fine to everyone else. Only your dashboard shows it.
• Our sample leans towards sites AI already cites. Sites that block ChatGPT wouldn't appear in ChatGPT's citations to begin with.
• Settings change. Cloudflare's defaults changed in September 2026, and hosting providers update theirs too.

These checks are one part of GEO, generative engine optimisation: improving how AI systems understand, cite and recommend a business. Once your site passes these five checks, the next question is whether ChatGPT actually names you. Request your free AI visibility check below to find out.