AI-native DeepTechDeepSEODeepAEOAI agentsGenerative searchHuman-approved

AI engines disagree with each other: why one citation is not a trend

By the SEOCells team, OmCells R&D Labs LLP · Published

Different AI engines often cite very different sources for the same question; the overlap between their top sources is near zero for many questions. The same engine also changes its answer from run to run. One citation in one engine is a direction, not a consensus. Measure several times, in several engines, and compare against normal variation.

Key points

  • ChatGPT, Google AI Overviews and AI Mode, Perplexity, Gemini and Claude each retrieve and choose sources in their own way.
  • For many questions, the engines' top sources barely overlap.
  • The same question asked twice in the same engine can give different sources.
  • Small week-to-week changes are usually noise. Judge a change against how much the numbers move when nothing changes.

Do the AI engines agree on who to cite?

Much less than people expect. Ask the same question in ChatGPT, Perplexity, Google's AI Overviews, Gemini and Claude and you will often see five different sets of sources. Published comparisons, and our own scans, find that the overlap between the engines' top sources is near zero for many questions. A handful of very large sites turn up across engines; most sources do not.

This makes sense once you remember that each engine searches its own way. Some lean on Google's index, some on Bing's, some on their own crawlers, and each has its own way of picking which passages to use. They are different products, built by different teams, with different ideas of what a good source looks like.

What does that mean for my business?

Say you run a small CRM startup, as an example. A customer tells you they found you through Perplexity, you check, and there you are, cited third. Great. But that tells you about Perplexity, for that phrasing, on that day. It says very little about ChatGPT, where your customers may spend more time, or about the slightly different question your next prospect asks.

So treat a citation in one engine as directional: a sign you are in the running, not proof that “AI” has settled on you. And when you plan work, decide which engines matter for your customers rather than trying to please all of them at once.

Why does the same engine give different answers?

AI answers are generated, not looked up. The wording of the answer, the searches the assistant runs behind the scenes and the sources it settles on can all shift between two runs of the identical question, minutes apart. Personalisation, location, the day's index and model updates add more movement on top.

If you check once on Monday, make a change, and check once on Friday, you cannot tell your change from that background movement. We see this often: an owner celebrates a new citation that would have appeared anyway, or panics over one that disappeared for a single run.

How should you measure, then?

  1. Fix a list of questions in your customers' words, and keep it the same over time.
  2. Pick the engines your customers actually use, and check each one separately.
  3. Ask each question several times per engine, not once, and record every source cited.
  4. Report how often you are cited (for example, “in some runs, not most”) rather than a yes or no.
  5. Before believing a change, look at how much your numbers move between two checks when you have changed nothing.
  6. Record which other sources are cited. They tell you more than your own count does.

Is a single citation worthless, then?

Not worthless, just thin. It proves your page can be read, retrieved and chosen by at least one engine for at least one phrasing. That is real information, especially if you were never cited before. What it cannot do on its own is show a trend, or show that something you did caused it.

Which engines should a small business watch?

Start with where your customers already are. A local trades business may find that Google's AI Overviews matter most, simply because its customers still start on Google. A software company selling to developers may care more about ChatGPT and Perplexity. Ask a few customers how they research before buying; the answer is usually more useful than any industry chart. Then watch two or three engines properly rather than five badly.

What about tools that give one “AI visibility score”?

A single number is convenient, but it hides how much the engines disagree and how much the answers move. Ask any tool, including ours, what a score is made of, which engines it covers, how many times each question was asked, and how wide the range around it is. A score without a range invites you to read noise as news.

The SEOCells AEO Report shows results per engine and gives ranges rather than single figures, so you can see where the engines agree and where they do not.

Questions answered

Why does ChatGPT cite different sources than Perplexity?

They search and choose sources in different ways, using different indexes and different selection methods. For many questions their top sources barely overlap.

Why did my business appear in an AI answer yesterday but not today?

AI answers vary from run to run, even for the same question. One appearance or disappearance is often ordinary variation. Check several runs before drawing a conclusion.

Should I optimise for one AI engine or all of them?

Start with the engines your customers use. The basics (readable pages, clear answers, mentions on trusted sites) help across engines, but results will still differ between them.

How many times should I ask the same question?

More than once per engine, and on more than one day. The aim is to see a pattern and a range, not a single snapshot.

Sources

  1. Google Search Central — AI features and your website
  2. Bing Webmaster Guidelines
  3. Aggarwal et al. (2024), the GEO paper, KDD 2024 (arXiv:2311.09735)

Search engines and AI assistants change often; check the linked documentation for the latest details. Nothing in this post is a promise of rankings, citations or results.

See how different AI engines treat your business, with ranges rather than snapshots.

See the AEO Report