AI crawler access
8/25
AI crawlers are locked out or ignored. That is the most expensive mistake on this list.
robots.txt
Your robots.txt is not cleanly retrievable (status no response). AI crawler access is therefore ungoverned.GPTBot
GPTBot is not mentioned in your robots.txt — access is ungoverned.ClaudeBot
ClaudeBot is not mentioned in your robots.txt — access is ungoverned.PerplexityBot
PerplexityBot is not mentioned in your robots.txt — access is ungoverned.Google-Extended
Google-Extended is not mentioned in your robots.txt — access is ungoverned.Live test
The homepage did not respond to our request at all, so the browser-versus-bot comparison is not meaningful.
Next step
Name GPTBot, OAI-SearchBot, ClaudeBot, PerplexityBot and Google-Extended explicitly in your robots.txt with "Allow: /" — and check your CDN or firewall so no bot rule intercepts them first.