← All articles

One AI Visibility Score Cannot Be True

Michael JacobsMichael Jacobs··6 min read
Two contrasting data trend graphs on a dark background, with one amber line declining sharply and the other rising, separated by a dotted horizontal reference line.

Between July and September 2026, Reddit's share of ChatGPT citations fell 94%. Over the same quarter its share of Perplexity citations rose 86%.

Both numbers describe the same site in the same period. Average them and you get something that happened to nobody.

The measurement published by a monitoring vendor

The figures come from Otterly.AI, published on 1 October 2026 and written by Rick Tousseyn. Otterly sells AI search visibility monitoring, which makes the finding more interesting rather than less, because it undercuts the simplest version of what the category sells.

Their figures cover three periods: 3 July to 7 August, then 8 to 31 August, then September. Each is Reddit's share of that engine's citations. Across the quarter that share fell 94% on ChatGPT and rose 86% on Perplexity.

Otterly ties the sharpest drop to a ChatGPT model rollout in August.

The two checks that make this more than noise

Volume is the obvious alternative explanation, and Otterly addresses it: by their account ChatGPT's total citations rose over the same period. ChatGPT did not stop citing. It stopped citing Reddit.

Concentration is the other. Otterly reports the decline in every industry it tracks, so no single sector carried the result.

So the decline was not a collection gap and it was not one vertical. Something changed inside one engine, and Reddit had no input into it.

Semrush saw the same instability on newly published pages

This matters because the Reddit case could be dismissed as a platform-specific policy move. Semrush's experiment looks at a different situation and finds the same instability.

In December 2025 Semrush published a 30-day test by Tushar Pol. They put 81 new FAQ-style pages on their own blog, a domain with an Authority Score of 84, and tracked citations daily. Best-case conditions, by their own description.

In Google AI Mode, 29 of the 81 pages were cited the day after publication. By day six, 48 pages. By day 14, 39. By day 30, 21.

Semrush does not say whether any of the pages were edited during the test, so read this as how citations of new content behave over a month, not as proof that identical pages rise and fall. Semrush's own conclusion: "What gets cited one week might not be cited the next."

ChatGPT behaved in the opposite direction over the same window, climbing from eight pages on day one to 34 on day thirty. Two engines, one set of pages, opposite trajectories.

What two studies together actually establish

Taken alone, each is a single finding from a vendor with a product to sell. Taken together they establish something harder to argue with.

Citation share moves for reasons the site does not control. It moves by large percentages over weeks. And it moves in different directions on different engines at the same time, which is the part that breaks the arithmetic.

Why this makes a blended score untrue rather than imprecise

An average is only meaningful when the things being averaged are commensurable. Here they are not, for three separate reasons.

The engines disagree about direction. A quarter that is catastrophic on ChatGPT and excellent on Perplexity has no honest single summary. Any weighting you pick is an editorial judgement about which engine matters, presented as a measurement.

Some engines may not use a given site as a source at all. Including a zero in an average implies the zero is a result. For a site an engine does not use as a source, it is not a result, it is a category error. Averaging it in drags the number down and makes it look like a problem you could fix.

The denominators are not stable. These are percentages of each engine's total citations, and that total is itself moving. By Otterly's account ChatGPT's citations rose while Reddit's share of them collapsed. Share and volume can move opposite ways, so a share figure alone does not tell you whether you were cited more or less often in absolute terms.

A score that hides all three is not a rounded version of the truth. It is a different claim.

Where Otterly's numbers need handling with care

Using this research properly means naming its limits. Four are worth stating.

Otterly is a vendor publishing research that supports buying monitoring from a vendor. That does not make the data wrong. It does mean the framing is not neutral, and the same applies to Semrush.

The sample is not published on that page. There is no prompt count, query set or geographic split alongside the table. Otterly documents its methodology separately and runs a public industry index, so the material exists, but it is not in the piece carrying the figures.

The comparison periods are unequal, at 36 days, 24 days and 30 days. For share figures that matters less than it would for counts, but it is not a clean quarter-on-quarter read.

And the causal claim is a date correlation. A drop coinciding with a model rollout is consistent with the rollout causing it. It is not a demonstration, and only OpenAI could supply one.

Reddit is also an unusual subject. It is one of the most heavily cited domains on the web and the object of commercial negotiation between platforms. A move of this size on a site like that does not automatically predict what happens to a mid-size brand.

What to measure instead

The practical conclusion is not that measurement is hopeless. It is that the unit has to match what moves.

Report per engine, never blended. If a stakeholder wants one number, give them the engine that matters to the business and say why that one.

Exclude engines that do not use you as a source, and state the exclusion. A zero that has held for a quarter is a coverage fact, not a performance figure.

Carry volume next to share. Share can fall while absolute citations rise. Without both, a line on a chart is ambiguous.

Attach a cause or label it unexplained. Otterly's sharpest drop coincides with a model release. Most movement will not have anything to point at, and writing "unexplained" is more useful than inventing a reason.

Sample enough to separate movement from noise before reporting either. A single daily reading cannot tell a trend from churn.

Separate what you control from what you do not

None of the volatility above was caused by anything a site owner did, and none of it was fixable by anything a site owner could do. That is the real lesson, and it has a practical edge.

Two things stay under your control. Whether an engine can reach, parse and lift an answer out of your page. And whether anything in that answer points back to you, through a named author, a resolvable entity, your own data, or a term that could only have come from you.

That is the distinction between retrievable and citable, and it is measurable directly on the page, today, repeatably, without asking an engine anything. It does not tell you whether you were cited. It tells you whether you are in a condition to be, which is the part that responds to work.

Measure citation share too, by all means. Just do it per engine, with volume beside it, and stop pretending a weighted average of disagreeing systems is a fact about your site.

Is AI visibility tracking worth paying for?+

It depends what you ask of it. As a per-engine trend over a long enough window, with a stable prompt set, it answers a real question. As a single blended score reported weekly, the evidence above says the number will move for reasons unconnected to anything you did.

Why do different AI visibility tools give me different numbers?+

Different prompt sets, sampling frequencies, geographies and engine coverage. Otterly's and Semrush's data both show large week-to-week movement in the underlying citations, so two tools sampling at different times can disagree without either being wrong.

If citations are this volatile, what should I optimise for?+

The conditions rather than the outcome. Access, structure, extractable answers, named authorship, resolvable entity markup and original evidence are all assessable on the page and stable over time. Citation share is downstream of model behaviour you do not control.

Does a 0% citation share on an engine mean I have a problem?+

Not necessarily. Some engines do not use some source types at all. Check whether the engine uses sources like yours before treating a zero as a failure.

Sources

The 94% and 86% figures are Otterly.AI's measurements, not ours. The Google AI Mode and ChatGPT page-citation figures are Semrush's.