- Google says there are no extra requirements to appear in AI Overviews or AI Mode. A page has to be indexed and eligible to show a snippet. Normal search basics apply.
- Each AI company has its own crawler. Blocking the wrong one can quietly remove you from its answers.
- Google-Extended controls Gemini training and some grounding. It does not affect Google Search or its ranking.
- Bing's index matters. It feeds Microsoft Copilot, and Microsoft made Bing the default search in ChatGPT in 2023.
- llms.txt is a proposal, not a standard. Google says you don't need AI text files to appear in its AI features.
- The durable advantage is the same as ever: clear answers and facts nobody else has.
Start with what is documented
A buyer types a question into ChatGPT. Something like "who can connect our field service software to our accounting system." The answer names three companies and links to two pages. Yours is not one of them.
The natural reaction is to look for a trick. There isn't much of one. What there is, is a short list of things the platforms have written down, and a long list of things people have guessed.
This guide keeps those apart. Where a platform documents something, we say so and link to it. Where it doesn't, we say that too.
What Google says about AI Overviews and AI Mode
Google's guidance for site owners is unusually direct. To be a supporting link in AI Overviews or AI Mode, a page must be indexed and eligible to be shown in Google Search with a snippet. Google says there are no additional technical requirements.
It goes further. Google says you don't need new machine-readable files, AI text files or special markup to appear in these features, and that no special schema.org structured data is required.
What it does recommend is ordinary search practice:
- Let Googlebot crawl the page, including through any hosting or security layer in front of your site.
- Link to important pages from other pages so they are easy to find.
- Keep important content in text, not only in images or video.
- Keep structured data consistent with what the page visibly says.
- Verify your site in Search Console and use it to diagnose problems.
Google also says traffic from AI Overviews and AI Mode is reported in the Search Console Performance report, under the Web search type. It is mixed in with everything else, not broken out.
If you want to limit what Google shows from your pages, the controls are the existing ones: nosnippet, data-nosnippet, max-snippet and noindex. They limit you in regular results too, so use them with care.
Know which crawler does what
Each AI company runs more than one crawler, and they do different jobs. Blocking one to keep your content out of training can, if you pick the wrong one, take you out of search answers.
- OAI-SearchBot is how OpenAI finds pages for ChatGPT search. OpenAI says sites that block it will not be shown in ChatGPT search answers, apart from bare navigational links.
- GPTBot is OpenAI's crawler for training. Blocking it tells OpenAI not to use your content to train its models. It is separate from search.
- ChatGPT-User visits a page when a person asks ChatGPT to. OpenAI says it is not used for automatic crawling and that robots.txt rules may not apply to it.
- PerplexityBot surfaces and links sites in Perplexity's results. Perplexity says it is not used to crawl content for AI foundation models.
- Perplexity-User fetches a page when a user's question calls for it. Perplexity says it generally ignores robots.txt, because a person asked for the page.
- Google-Extended is not a separate crawler. It is a robots.txt token that controls whether Google may use your content to train Gemini models and for grounding in Gemini Apps and some other Google systems. Google says it does not affect inclusion in Google Search and is not a ranking signal.
That last point surprises people. Blocking Google-Extended is a choice about training and some Gemini uses. It is not a way to stay out of AI Overviews, and it does not hurt your rankings.
Then check the layer nobody thinks about. Some hosting and security services can block AI crawlers with one switch, and some turn that on by default for new sites. Your robots.txt can say "welcome" while your firewall says "no." Look at both.
Why Bing matters more than you'd guess
Most teams watch Google and ignore Bing. For AI answers, that is a mistake.
Microsoft Copilot draws on Bing. Bing Webmaster Tools now has an AI Performance report, which Microsoft says shows how your content is cited across Copilot, AI summaries in Bing and select partner experiences.
ChatGPT has a long history with Bing too. In May 2023, Microsoft announced Bing as the default search experience in ChatGPT. ChatGPT search has changed since then, and OpenAI's publisher FAQ refers to third-party search providers without listing them all. So treat Bing as one well-documented route into ChatGPT, not the whole story.
The practical steps are cheap. Verify your site in Bing Webmaster Tools. Submit your sitemap. Consider IndexNow, a protocol that tells participating search engines when a page changes. Its own documentation lists Bing, Yandex, Naver, Seznam, Yep and Amazon. Google is not on that list.
Microsoft is also changing its controls. In October 2026 it announced separate preferences for AI training and AI grounding, with the old noarchive and nocache tags no longer read that way after March 1, 2027. If you use those tags today, read the announcement before then.
Write answers a machine can lift
AI answers are built from pieces of pages. A page that states its answer plainly, near a clear heading, is easier to use than one where the answer is spread across four paragraphs and a PDF.
Microsoft's own guidance on content for AI answers says much the same. Keep the search basics. Match your title, main heading and description. Use descriptive headings. Don't hide key answers in tabs, images or PDFs. Write sentences that still make sense when lifted out on their own.
A simple test helps. Pick the five questions your buyers ask most on a first call. For each one, find the sentence on your site that answers it. If you can't find it, or it only exists inside a downloadable brochure, that is your first job.
Specific beats general. "We work with companies in many industries" answers nothing. "We connect ServiceTitan to QuickBooks Online and Sage Intacct" answers a real question someone will ask.
Structured data: useful, not magic
Structured data labels what a page is about: an organization, an article, a product, a set of questions and answers. Google says it helps it understand pages and can make them eligible for rich results. It does not guarantee them.
Google is firm on one rule. The markup must describe what visitors can see. Don't mark up information that isn't on the page, even if it's true.
For AI features, Google says no special structured data is required. So add the standard types that fit, keep them accurate, and check them with Google's Rich Results Test. Then move on to the content.
Original facts are what get cited
An AI answer that pulls from ten pages saying the same thing has no reason to cite yours. It cites the page with something the others don't have.
Google's guidance on helpful content asks whether a page offers original information, reporting, research or analysis. Microsoft's Bing team has written that answers need clear sources and current facts, and that stale facts produce wrong answers.
For most companies, the original material is already in the building. How long a typical project really takes. Which systems you connect and what each one allows. The questions customers ask before they buy. Write those down, accurately, and keep them current.
What about llms.txt?
llms.txt is a plain text file at the root of a site that summarizes it for language models. It was proposed by Jeremy Howard in September 2024, and its own site still calls it a proposal.
None of the major engines we've covered here has documented using it. Google says you don't need AI text files to appear in its AI features. We publish one on our own site because it is cheap and harmless. We don't expect it to do the work that clear pages do, and neither should you.
Measuring it honestly
Measurement is patchy, and anyone selling precise numbers is filling gaps with estimates.
- Google: AI Overviews and AI Mode traffic sits inside the Search Console Performance report, not separately.
- Microsoft: the AI Performance report in Bing Webmaster Tools shows citation counts and sample grounding queries.
- ChatGPT: links from ChatGPT often carry utm_source=chatgpt.com, so your analytics can show many of those visits. Not every link is tagged.
- Everyone else: referrals from perplexity.ai, copilot.microsoft.com and similar sites show up in analytics as ordinary referrers.
Watch the trend and the inquiries, not a single score.
What most guides miss
The biggest wins are often removals. A firewall rule blocking OAI-SearchBot. A noindex tag left over from a staging site. A key page that only exists as a PDF. Fixing what stops you being read beats any new tactic.
Training and answers are different decisions. You can decline model training and still appear in search answers. OpenAI, Perplexity, Google and now Microsoft all separate the two. Decide each one on purpose.
Being cited is not the same as being chosen. An answer that mentions you sends a buyer to your site with a question already half answered. The page they land on still has to make the next step obvious.
Accuracy compounds. A wrong fact on your site can be repeated in answers for months. Review dates on pages, and the habit of actually rechecking them, matter more than they used to.
How we do it
We start with a free intro call and a look at what blocks your site from being read: crawler rules, security settings, missing pages. Then we work alongside your team to write the pages that answer your buyers' real questions, with every fact checked. Our own site is built this way. See websites and content for how the work runs.
Sources
- Google Search Central: AI features and your website
- Google Search Central: Google's common crawlers (Google-Extended)
- OpenAI: Overview of OpenAI crawlers
- Perplexity: Perplexity crawlers
- Bing Search Blog: Bing at Microsoft Build 2023
- Bing Webmaster Blog: Introducing AI Performance in Bing Webmaster Tools
- Bing Search Blog: Clearer preferences for training and grounding (October 2026)
- Bing Search Blog: Evolving role of the index
- Microsoft Advertising: Optimizing your content for inclusion in AI search answers (October 2025)
- IndexNow: FAQ
- Google Search Central: Introduction to structured data markup
- Google Search Central: Creating helpful, reliable, people-first content
- llmstxt.org: The /llms.txt file
