How to tell, honestly, whether a change made a difference to your AI visibility

By the SEOCells team, OmCells R&D Labs LLP · Published 8 November 2026

To judge a change honestly, wait a few weeks, since first citations usually take that long. Measure before and after with the same questions in the same engines. Compare with similar sites, so an engine-wide shift does not fool you. Read ranges, not single numbers. And accept “no clear effect yet” as a real answer.

Key points

AI answers vary run to run, so one before and one after check proves almost nothing.
Keep the questions, engines and method identical before and after.
Engines change too. Compare against similar sites you did not change.
A range that overlaps “no change” means you do not know yet, and that is fine.

Why is this harder than it sounds?

Imagine an online shoe shop that rewrites its sizing guide on the 1st, checks ChatGPT on the 5th and finds itself cited for “do these trainers run small?” It is tempting to call that a win. But the answer might have cited the shop anyway on the 5th. The engine may have updated that week. A competitor's page may have gone offline. Three different explanations, and a single check cannot tell them apart.

We see this often. Not because people are careless, but because AI answers move around more than anyone expects, and the human brain is very good at finding a story in noise.

How long should you wait?

A few weeks at least. The engine has to recrawl the page, its search index has to pick up the change, and the answers have to start drawing on it. In our experience, first citations after a change usually take a few weeks to appear, and changes that rely on other sites mentioning you take longer. Checking after three days mostly measures your patience.

What does a fair before-and-after look like?

Write the question list before you make the change, and do not edit it afterwards.
Use the same engines, the same location settings and, as far as possible, the same way of asking.
Ask each question several times per engine, both before and after, and record every source cited.
Note the dates of your change and anything else that changed at the same time.
Measure on several days before the change, so you know how much the numbers wobble on their own.

Why compare with other sites?

Because engines change underneath you. If an engine starts showing more sources per answer, or leans more on a particular kind of site, everyone in your market may see their citations rise or fall at once. If you only watch yourself, you will credit or blame your change for something that happened to everyone.

So pick a few comparable sites, for example competitors of similar size whose pages you did not change, and track them alongside yourself. If you went up and they stayed flat, that is more interesting than if everyone went up together.

What do ranges tell you?

A single figure such as “cited in 4 of 10 runs” sounds precise, but with so few runs the true rate could be quite a bit lower or higher. A range shows that uncertainty. If the range after your change overlaps the range before it, you cannot say the change made a difference yet. If the ranges clearly separate, and comparable sites did not move in the same way, you have something worth believing.

Is “no clear effect yet” a failure?

No, it is an honest result. Many sensible changes have small effects that take time to show, and some have none on citations but still help the people reading your page. Reporting “no clear effect yet” protects you from two expensive mistakes: abandoning something good too early, and doubling down on something that only looked like it worked because of noise.

Be wary of anyone, an agency or a tool, that reports a clear win after every change. Real measurement does not look like that.

What should you record each time?

Keep it simple. A row per question, per engine, per run: the date, whether you were cited, where you sat among the sources, which other sources were cited, and a short note if the answer said something wrong about you. Add a separate log of changes, yours and anything you notice in the market. Months later, that log is what lets you explain a jump or a dip instead of guessing.

Can this be done for you?

Yes. SEOCells' Deep Scan includes “Measure the impact of a change”: run it on a page you scanned before, a few weeks after your change, and it compares before and after for the same questions and engines and reports the result as a range, including “no clear effect yet” when that is the honest answer. You can also do it by hand with the steps above; it just takes discipline and a spreadsheet.

Questions answered

How soon after a change can I see results in AI answers?

Usually not for a few weeks. Engines need to recrawl and re-index the page, and changes that rely on other sites mentioning you take longer.

Why did my citations go up even though I changed nothing?

AI answers vary from run to run, and engines change how they choose sources. That is why comparing with similar sites and measuring several times matters.

How many checks do I need before and after?

Several per question and engine, on more than one day, so you can see how much results move on their own. One check before and one after is not enough.

What does “no clear effect yet” mean?

That the results before and after overlap too much to tell them apart. It is a real result, not a failure, and it may change as more time passes.

Sources

Search engines and AI assistants change often; check the linked documentation for the latest details. Nothing in this post is a promise of rankings, citations or results.

Measure the impact of a change with ranges and a comparison against similar sites.

See how Deep Scan works