How AEO agencies prove results, and what a monthly report has to show

Every AEO pitch arrives at the same question, and most of them arrive at it badly. You are being asked to pay for a number that the vendor also produces. If the same party sets the questions, runs them, and reports the score, the report is not a measurement — it is a marketing asset with a table in it.

So the useful thing to establish before you sign is not whether an agency is confident. It is which of its claims you could check yourself, today, without its cooperation.

How do AEO agencies prove their results?

Three ways, and they are not equally checkable. SearchD sorts vendor evidence by a single test: how much of the claim can you verify before you pay for it?

The first row is the strongest of the three when the introduction comes with it, and SearchD tells buyers to ask for it first — of every vendor, itself included. What decides whether it is evidence is the baseline behind it. A named rate means nothing except against its own before-state, and that is the one thing nobody can reconstruct later: once a change has shipped, the before is gone. So the two questions to put to a reference are which panel was used and whether it was frozen before the work began. A case study that survives both is worth more than anything else on this page. One with no client you can reach is a claim wearing a number.

That same logic is why the third row has to be published before the selling starts rather than after a good month.

Google’s own guidance is the reason to be strict here. It states plainly that no third-party tool has insight into its ranking or AI systems (Google, AI features and your website). Any vendor promising you a placement inside an answer is promising a mechanism nobody outside the engine can see.

What should a monthly AI visibility report show?

Six sections, on one page, and a report missing any of them is hiding something structural rather than saving you reading time. This is the shape SearchD publishes, including the sections that make it look worse.

Section 1 is per engine for a reason. The engines do not read the same index — Claude’s web search runs on Brave’s, Google’s surfaces read Google’s, and ChatGPT serves from pipelines including one of its own, identified by instrumenting 1,200 answers where only 1.5% of the URLs appeared in Bing’s top 20 for the same queries (Search Engine Land, 17 August 2026). A single blended percentage averages four separate facts into one that describes none of them.

Section 5 is where most reports go quiet. If the report does not say what was shipped and when, no delta in it can be attributed to anything, and the agency has reserved the right to claim every rise. Model updates move these numbers too, in both directions, and nobody can fully separate the two — SearchD records the model version on every observation and re-runs within a week of a major release, and where a jump is plainly a model update the report says so.

Why does an AI visibility report need a noise floor?

Because the same question asked twice does not reliably return the same answer, so a small movement is not evidence of anything until you know how big the instrument’s own wobble is. SearchD measures that floor from a stability run — the same panel, repeated with nothing changed — and reports every delta against it.

The discipline that matters is symmetry. A +2 and a −1 inside the same ±3 band are both noise, and an agency that marks the +2 as progress while calling the −1 noise has told you the floor is decorative. SearchD reports named rate per engine at n=40, which is the basis the floor is derived from; the arithmetic behind reading such a number is in what a good named rate is.

Is there an agency that publishes its own AI visibility numbers before selling AEO?

SearchD does, and the number is currently zero. In the pilot reading of 8 September 2026 — 17 buyer questions asked on ChatGPT, Claude and Gemini, 51 observations — no engine named SearchD on any of them. The figure is on /results and goes up monthly, including the months it goes the wrong way.

A new domain with no corpus should be at zero, so a launch number that is anything else is the thing worth interrogating. The monthly panel SearchD reports against is 40 questions across five engines — ChatGPT, Perplexity, Claude, Gemini and Google AI Overviews — frozen in September 2026 and unchanged from here.

The same standard is what SearchD asks buyers to hold every vendor to, itself included. The full list of questions to put to an AEO vendor, with SearchD’s own answers filled in including the three it loses, is on the evidence page.

Which proof claims should make you walk away?

Five, and each one fails against a source you can read rather than against SearchD’s opinion. These are claims SearchD checked before deciding what it would sell, and declined to sell.

The last row is the one that most often survives a procurement review unchallenged, because it sounds like the SEO deliverable everyone already knows how to buy. It fails because the engine does not search the sentence your buyer typed — it generates several searches of its own and answers from what those return. What ChatGPT actually searches before it names a brand covers that mechanism and the order of work that follows from it.

How do you check an agency’s proof in ten minutes?

Four steps, none of which require the agency’s cooperation, and all of which you can run before the second call.

1. Ask for a client you can call, and for the agency’s own number. Both, not either. If there is no reference, ask what the agency’s own named rate was last month and on what date — an agency selling measurement that does not measure itself has answered the question.
2. Ask who holds the question list. If the panel is proprietary, the denominator is hidden and the number cannot be audited. You should receive the questions, and they should be frozen before any work begins.
3. Ask what the noise floor is and how it was measured. Then look at last month’s report and check whether a negative delta inside that band was also called noise.
4. Ask what happens if it does not work. The answer should name something the agency controls and point at a clause in the agreement, not at a page on its website.

A vendor that answers all four cleanly may still not move your number — engine behaviour is nobody’s to promise. But you will at least be buying a measurement rather than a narrative, and you will know on which month to stop.

What that measurement is worth on your own brand depends on whether there is a quotable passage for an engine to lift once it can reach you, which is a separate piece of work: what makes a passage get cited. The monthly loop SearchD runs against the panel above is described on the retainer page, and where a monitoring tool is the better purchase is set out on alternatives.