Free tool
AI bot access checker
AI assistants can only recommend pages their crawlers can reach. Paste any page from your site: we check robots.txt rules for 16 AI and search crawlers, Cloudflare content signals and noindex, and whether requests that identify as AI crawlers get blocked.
Checking…
How it works
- 01
Paste a page from your site
A product page is best: rules can differ by path.
- 02
We read robots.txt
Following RFC 9309: the most specific user-agent group wins, then the longest matching rule.
- 03
We send test requests
As a browser and with AI crawler names, and compare the answers.
What you get
-
AI search crawlers
OAI-SearchBot (ChatGPT search), PerplexityBot, Claude-SearchBot, Googlebot (AI Overviews and AI Mode), Bingbot (Copilot), Applebot.
-
Visits shoppers ask for
ChatGPT-User, Perplexity-User, Claude-User: when a shopper pastes your link into an assistant.
-
Model training crawlers
GPTBot, ClaudeBot, Google-Extended, Applebot-Extended, CCBot, Meta-ExternalAgent. Blocking them is a business choice and doesn't remove you from AI search.
-
Bot walls and signals
Cloudflare challenges, firewall blocks, Content-Signal (ai-input, ai-train), noindex and nosnippet.
Quelta product videos
Turn the open questions into product videos
Quelta makes product-accurate videos from your product photos, for a whole catalog: the product looks the same in every shot. Each video answers one buyer doubt.
See product videosStart with a pilot
Tell us about your catalog
Send a link to your catalog or website and tell us roughly how many videos you need a month. We start with a pilot on 3–5 of your products, so you can judge the videos on your own catalog.
Send a link to your catalog
Your website, a product feed or your store on a marketplace.
We make a pilot on 3–5 of your products
Product-accurate videos of your real products, for the markets and platforms you need.
A monthly contract for the volume you need
New videos for the catalog and new arrivals every month, in one style.
We don’t copy other people’s videos. Every video is new and about your product. We keep the mechanics that sell (script, pacing, delivery), not anyone’s footage, people or music.
Contact us
Made with our engineTests, not client projects
Questions and answers
If I block GPTBot, will I disappear from ChatGPT?
No. GPTBot collects data for model training. ChatGPT search uses OAI-SearchBot, and visits a user asks for use ChatGPT-User. Block training if you want, but keep the search and user agents allowed on product pages.
Does Cloudflare block AI crawlers?
Since July 2025 Cloudflare blocks AI crawlers by default on new domains, and its managed robots.txt can add content signals. Check Security → Bots and AI Crawl Control in your Cloudflare dashboard.
Does blocking Google-Extended remove me from AI Overviews?
Google-Extended controls the use of your content in Gemini apps and model training. Google Search, AI Overviews and AI Mode use Googlebot, so a nosnippet or noindex tag is what keeps a page out of them.
Why test with crawler names if real crawlers use their own IPs?
Many firewall rules match the user agent string, so a block on a request with a crawler's name usually means the real crawler is blocked too. It's a strong hint, not proof.