# Showing up in ChatGPT, Claude and Perplexity: what a small business controls > Which AI crawlers to let in, how to write robots.txt and what content AI engines can actually cite. A practical guide for Quebec service businesses. Updated on October 15, 2026 · https://app.pilotiko.com/en/guides/show-up-in-chatgpt-and-ai-answers To have a chance of being cited by ChatGPT, Claude or Perplexity, allow their search crawlers (OAI-SearchBot, Claude-SearchBot, PerplexityBot) in robots.txt and publish verifiable pages that answer your customers' questions plainly. Blocking training crawlers does not remove you from their search answers. No setting can promise you a citation. ## In short - Each AI company splits its crawlers into search, training and user-triggered visits. - Blocking OAI-SearchBot removes your site from ChatGPT search answers; blocking GPTBot does not. - Google-Extended has no effect on where you appear in Google Search. - Cited content tends to be precise, dated, sourced and easy to quote. ## Which AI crawlers visit a small business website? There are three kinds: crawlers that feed the assistant's search feature, crawlers that collect content to train models, and visits triggered by one user's question. OpenAI documents three agents. OAI-SearchBot surfaces sites in ChatGPT's search features, GPTBot gathers content that may be used to train models, and ChatGPT-User fetches a page when a user asks for it. OpenAI states that each setting is independent of the others [1]. Anthropic follows the same pattern with ClaudeBot (training), Claude-User (fetches on behalf of a Claude user) and Claude-SearchBot (indexing for search) [2]. Perplexity separates PerplexityBot, which surfaces and links sites in its results, from Perplexity-User, which fetches pages on request [3]. Google has no separate crawler for AI Overviews: it's Googlebot. The Google-Extended token only controls whether your content may train Gemini or ground its answers outside Search. Google says it does not affect inclusion in Google Search and is not a ranking signal [4]. | Crawler | Company | Purpose | If you block it | |---|---|---|---| | OAI-SearchBot | OpenAI | ChatGPT search | Your pages stop appearing in search answers (navigational links still possible) | | GPTBot | OpenAI | Model training | No effect on ChatGPT search | | ChatGPT-User | OpenAI | User-requested visit | Not used to decide search inclusion | | Claude-SearchBot | Anthropic | Search indexing | May reduce visibility in answers | | Claude-User | Anthropic | User-requested visit | May reduce visibility | | ClaudeBot | Anthropic | Training | Future content excluded from training | | PerplexityBot | Perplexity | Perplexity results | Less chance of being linked; not used to train models | | Perplexity-User | Perplexity | User-requested visit | Generally ignores robots.txt | | Google-Extended | Google | Gemini training and grounding | No effect on Google Search | ## What goes in robots.txt if you want citations but not training? Allow the search crawlers and, if you choose, disallow the training crawlers. That's a business decision, not a technical requirement. Plenty of owners block everything after reading one alarming article about AI. The result is a site that vanishes from ChatGPT answers when the real goal was only to stay out of training data. Open yoursite.com/robots.txt and read what is already there before touching it. Anthropic notes that the rule has to be set on every subdomain you want covered [2]. If your booking page lives on a subdomain, it needs its own file. **Example: robots.txt for a service business that wants to be cited** A made-up file to adapt. Disallowing the training crawlers is optional; it doesn't change your visibility in ChatGPT, Claude or Google search. - User-agent: OAI-SearchBot : Allow: / - User-agent: Claude-SearchBot : Allow: / - User-agent: PerplexityBot : Allow: / - User-agent: GPTBot : Disallow: / - User-agent: ClaudeBot : Disallow: / - User-agent: Google-Extended : Disallow: / - User-agent: * (everyone else) : Disallow: /account/ (private pages) - **= 9 working lines: search open, training declined, customer area closed.** ## What kind of content do AI engines prefer to cite? Content that can be checked and quoted: a clear answer up top, dated and sourced numbers, an identified business behind it, and facts that are specific to you. Google asks for "non-commodity" content: first-hand experience and expertise that go beyond common knowledge. It also says there's no need to chop pages into small chunks or rewrite them for AI [5]. What helps an AI engine is what helps a human reader. For a service business, that means your actual service area, real lead times, starting prices if you publish them, licences (RBQ, for example) and answers to the questions customers ask on the phone. A three-line generic "Our services" page gives an AI nothing specific to quote. - A 40 to 60 word answer under each important heading. - Headings phrased like your customers' questions; see [how to write a useful FAQ](https://app.pilotiko.com/en/guides/write-an-faq-that-ranks). - Accurate structured data about the business: [the structured data guide](https://app.pilotiko.com/en/guides/structured-data-for-small-business). - An up-to-date Google profile, which Google also draws on in its answers: [completing your Google Business Profile](https://app.pilotiko.com/en/guides/complete-google-business-profile). - City pages only when they carry real local information: [local SEO in Quebec](https://app.pilotiko.com/en/guides/local-seo-quebec). ## Do you need an llms.txt file? Not for Google, possibly for other tools. It's a 2024 proposal, not a standard, and Google Search doesn't use it. llms.txt is a 2024 proposal: a Markdown file at the site root that summarizes the business and points to key pages [6]. Google says Search doesn't use it, that it neither helps nor hurts visibility, and that you can keep it for other services [5]. If it's generated automatically, the cost is low. The real risk is an llms.txt that still lists services you dropped last year. Keep it in sync with the site. ## How can you tell whether an AI already cites your business? Once a month, ask ChatGPT, Claude and Perplexity the ten questions customers ask you most, and note who gets cited. The full method, with a tracking table, is in [measuring your AI visibility](https://app.pilotiko.com/en/guides/measure-ai-visibility). The [AI presence tool](https://app.pilotiko.com/en/tools/ai-presence-test) helps you set up that question set. For the big picture (technical SEO, direct answers, Google profile and measurement), read the pillar guide on [getting found on Google and cited by AI](https://app.pilotiko.com/en/guides/google-and-ai-visibility). ## What does Pilotiko do for AI visibility? When Pilotiko builds and hosts your site (website), it generates robots.txt, the sitemap, structured data and llms.txt, and it can audit your current site (SEO, AEO and GEO). The robots.txt rules stay your call: you see what each line does before anything is published. > Coming soon: automatic IndexNow submission of changed pages. It isn't available yet. ## Frequently asked questions ### If I block GPTBot, do I disappear from ChatGPT? No. OpenAI says blocking GPTBot only signals that your content shouldn't be used for training. OAI-SearchBot is the crawler that governs whether you show up in ChatGPT search answers. ### Does blocking Google-Extended hurt my Google rankings? No. Google states that Google-Extended doesn't affect a site's inclusion in Google Search and isn't used as a ranking signal. It only covers Gemini training and grounding. ### Can I pay to be cited by ChatGPT or Perplexity? None of the documentation we reviewed describes a paid way to be cited in answers. Be wary of anyone selling citations: Google itself notes that no third-party tool has access to its internal systems. ### My site is new. How long before AI engines cite it? No one publishes a timeline. Your pages first need to be crawled and indexed by each engine, then judged relevant to a question. Check indexing in Search Console and Bing Webmaster Tools first. ### Does Perplexity respect robots.txt? According to Perplexity, PerplexityBot follows robots.txt. Perplexity-User, which fetches a page because a person asked, generally ignores it since a human triggered the visit. ### Should I write differently for AI? Google says no: there's no special writing style for generative AI search. Write clear, precise answers that are specific to your business; that serves customers and engines alike. ## Sources 1. OpenAI, « Overview of OpenAI Crawlers », https://developers.openai.com/api/docs/bots (accessed on 2026-10-15) 2. Anthropic, centre d'aide Claude, « Does Anthropic crawl data from the web, and how can site owners block the crawler? », https://support.claude.com/en/articles/8896518-does-anthropic-crawl-data-from-the-web-and-how-can-site-owners-block-the-crawler (accessed on 2026-10-15) 3. Perplexity, « Perplexity Crawlers », https://docs.perplexity.ai/docs/resources/perplexity-crawlers (accessed on 2026-10-15) 4. Google, « Google's common crawlers (including Google-Extended) », https://developers.google.com/crawling/docs/crawlers-fetchers/google-common-crawlers (accessed on 2026-10-15) 5. Google Search Central, « Optimizing for generative AI search », https://developers.google.com/search/docs/fundamentals/ai-optimization-guide (accessed on 2026-10-15) 6. llmstxt.org, « The /llms.txt file (proposition) », https://llmstxt.org/ (accessed on 2026-10-15)