We checked if AI can read the world's biggest websites. 29% block it.

More and more people ask ChatGPT, Perplexity and Google's AI answers instead of scrolling through ten blue links. That raises a simple question almost nobody checks: can those AI engines actually read and cite your website?

So we ran the test. Using SEOWrench's own engine, we audited 32 of the most-visited websites — news, SaaS, e-commerce, developer and Brazilian sites — for GEO (Generative Engine Optimization) readiness. Here's what we found, with the method laid bare so you can check us.

The headline: many top sites are invisible to AI answers

AI crawlers are governed by your robots.txt, and there's a subtle trap — the bots that feed live answers (OAI-SearchBot, PerplexityBot, ChatGPT-User) are different from the ones that feed model training (GPTBot, ClaudeBot, Google-Extended). Block the first group and you can't be cited, no matter how good your content is.

Nearly 1 in 3 of the world's biggest sites can't appear in AI answers — by blocking the bots that generate them.

29% block the answer bots → they cannot be cited by ChatGPT or Perplexity.
42% block at least one AI bot in total.
59% expose no schema that helps an AI interpret the page.
Only 7 of 31 were fully ready — not blocking AI, readable without JavaScript, and with relevant schema.

News publishers are at war with AI

The blocking isn't random. Break it down by category and one group stands out:

Share of sites blocking AI answer bots, by category. Publishers block on purpose; most businesses don't block at all.

6 of 8 news sites block the answer bots: The New York Times, BBC, The Verge, TechCrunch, Wired and CNN. That's a deliberate business decision — publishers don't want to feed AI engines their journalism for free. It's a choice, not a mistake.

Businesses and SaaS companies are the opposite. They want to be cited, and almost none block AI (just 1 in 10). Their problem is quieter: over half expose no structured data to help an AI understand the page — and, crucially, most have never checked their status at all.

Who got it right

Seven sites passed every technical GEO signal we measured — not blocking AI, content readable without JavaScript, and relevant schema present:

Fully ready
Stripe	HubSpot
Vercel	Mailchimp
Dropbox	Nike
G1 (globo)

An honest caveat about JavaScript

You'll often hear that "AI can't read JavaScript sites." It's true that ChatGPT, Claude and Perplexity do not run JavaScript — they read the raw HTML. But in this sample, zero sites were JS-blank, because sites this big already server-render (SSR) their content.

The JavaScript trap is real, but it mostly bites smaller sites and startups that ship a client-rendered SPA. It deserves its own study — and we won't pretend our big-site sample proves anything about it. (This honesty is the whole point: a page an AI can't read is invisible, and we'd rather tell you the truth than inflate a number.)

Method, in the open

Sample: 32 of the most-visited sites across news, SaaS, dev, e-commerce and Brazil. Intentionally biased toward large sites — this is not a random sample of the web.
Blocking: read from each site's robots.txt; "blocks" means Disallow: / for that bot's user-agent. We split answer bots from training bots.
Readability & schema: we analysed the raw HTML of each homepage — exactly what a non-rendering AI crawler sees.
Coverage: robots.txt for 31/32, homepage for 32/32.
Validation: we manually re-checked the robots.txt of NYT, Figma and Reuters against our engine's output — a 100% match.

The takeaway

The point isn't that these sites are "doing it wrong." Some giants block AI on purpose; many other companies block it — or become unreadable — without realising. The real problem is that almost nobody knows their status.

You should. We built SEOWrench to answer exactly this, per engine, in seconds — and to be honest about what's technical versus what depends on content and authority. If you want the full picture of how the GEO score works, read how SEOWrench measures GEO.

Run a free audit and see whether ChatGPT, Claude, Perplexity, Gemini and Copilot can read and cite your site.

Sources & data

The raw dataset behind this post (collected 2026-07-10 with SEOWrench's own engine).
OpenAI — bot documentation (GPTBot vs OAI-SearchBot vs ChatGPT-User): developers.openai.com/api/docs/bots
Vercel — The rise of the AI crawler (AI crawlers don't execute JavaScript): vercel.com/blog/the-rise-of-the-ai-crawler