Every AI visibility tool on the market sells a score. Ours included. The uncomfortable question the whole category avoids is simple: does the score actually predict anything you can observe in the real world?

The stakes are not academic. ChatGPT alone reports 800 million weekly active users, and according to Pew Research, when an AI summary appears, users click traditional results roughly half as often (8% vs 15% of visits). In that world, being cited inside the answer is the visibility that remains.

We decided to find out, using the one dataset nobody else has: our own. Kuroma runs continuous visibility scans that ask seven AI engines real customer questions and record every answer and every citation. That gave us 201,695 real AI answers collected over 22 weeks, with 869,783 extracted citations, sitting next to the audit scores each brand held before those answers were generated. This article is a case study in measuring your own product honestly: our team regressed one dataset against the other and reports everything, including the parts that did not flatter us.

Key takeaways

  • Kuroma's AI Readiness factor scores predict whether a brand's own domain gets cited by ChatGPT with a held-out AUC of 0.84, where 0.50 is chance and 1.00 is perfect ranking.
  • Grok follows at 0.76, Claude at 0.68, and the Google surfaces (AI Overviews, AI Mode) plus Perplexity land between 0.61 and 0.63.
  • Gemini almost never cites a brand's own domain in our corpus (base rate near 0.2%). It cites retailers, press, and review sites instead, so own-domain citation is the wrong measure of success for that engine.
  • Predicted citation rates track observed rates across risk bins, so the score is usable as a probability, not just a ranking.
  • We are deliberately NOT reweighting our rubric from this model's coefficients. At 17 training brands, individual factor weights absorb brand-fame effects that have nothing to do with causality. Discrimination is proven; attribution is not.

Either the engines cite you or they do not. If a readiness score cannot separate brands that get cited from brands that do not, on data it has never seen, it is decoration.

Why does an AI visibility score need validation?

Generative engines answer questions by synthesizing multiple sources and citing a handful of them, a dynamic formalized in the academic literature on generative engine optimization (Aggarwal et al., the paper that named the field, showed visibility in generated answers can shift dramatically depending on how content is presented). An audit score claims to compress "how ready is this site to be cited" into a number.

But a claim like that is testable, and the test is brutal in a way traditional SEO audits never faced: the outcome is observable. Either the engines cite you or they do not. If a readiness score cannot separate brands that get cited from brands that do not, on data it has never seen, it is decoration.

We searched for any published out-of-sample validation of an AI visibility audit score against observed citations. We found none, from any vendor. The closest published work is correlational: Ahrefs, to its credit, has published large-scale correlation studies of which site traits co-occur with AI visibility, but a correlation measured on the same data it describes is a different claim from predicting citations the model has never seen. So we ran the out-of-sample test ourselves in July 2026, and we are publishing the method, the numbers, and the limits together.

How did we run the test?

Three rules kept the test honest.

First, labels are distributions, not single runs. AI answers are probabilistic: the same question produces different citation sets run to run, an inconsistency well documented across engines. In our analysis, we aggregated outcomes per brand, per engine, per ISO week, over 201,695 simulation runs. A brand-week is the unit of observation, and each one carries the number of runs and how many of those runs cited the brand's own domain.

Second, the score must come from the past. Every brand-week of outcomes was paired with the audit score that brand held strictly before that week began. We verified this row by row: zero future leakage. A model that peeks at the week it is grading is worthless.

Third, validation is out of sample in time. We fit a regularized logistic regression on earlier weeks and scored it only on later weeks it never saw. The reported metric is AUC on those held-out weeks: the probability the model ranks a cited case above an uncited one. 0.5 means no better than chance.

The full factor definitions and weights we ship are public on our methodology page, which now carries this validation as a standing section.

What did we find, engine by engine?

Horizontal bar chart of held-out AUC by engine: ChatGPT 0.84, Grok 0.76, Claude 0.68, Google AI Overviews 0.63, Perplexity 0.61, Google AI Mode 0.61. Baseline 0.50 equals chance.
Held-out AUC by engine, volume cohort (17 brands, 175 brand-week observations).
Engine Held-out AUC Reading
ChatGPT 0.84 Strong: the audit ranks cited above uncited brand-weeks 84% of the time on unseen weeks
Grok 0.76 Solid signal
Claude 0.68 Moderate
Google AI Overviews 0.63 Weak but above chance
Perplexity 0.61 Weak but above chance
Google AI Mode 0.61 Weak but above chance
Gemini excluded See the next section

Two details matter more than the headline, and both are key findings in their own right.

Notably, the result is stable under regularization: sweeping the penalty strength across a 16x range moves ChatGPT's held-out AUC between 0.84 and 0.86. And a second, thinner cohort of 7 brands collapsed to chance-level performance out of sample, exactly what small-sample theory predicts. We report that cohort's failure with the same prominence as the success, because a validation pipeline that only ever produces good news is not a validation pipeline.

How well calibrated are the predictions?

Ranking is one thing; probabilities are another. We grouped held-out ChatGPT brand-weeks into three bins by predicted citation rate and compared prediction to reality.

Three stat callouts comparing predicted and observed citation rates: predicted 1.9% observed 2.6%; predicted 17.3% observed 22.4%; predicted 75.9% observed 75.0%.
Predicted vs observed own-domain citation rates on held-out weeks, ChatGPT.

The noteworthy part is calibration at the extremes: a brand-week the model called roughly 76% citable was observed at 75%. The low-risk bin predicted at 1.9% came in at 2.6%. For a first pass on modest data, the score behaves like a probability, which is what makes it operationally useful: it tells you not just where you rank but how far you are from the citation threshold.

Why does Gemini almost never cite brand domains?

The strangest thing we found was not about our score at all. Across the entire corpus, our research shows Gemini's rate of citing a brand's own domain was effectively zero: 0.2% in one period, 0.0% in another, across hundreds of thousands of runs.

Gemini answers cite plenty of sources. They are just not yours. Retailers, press coverage, review sites, and aggregators dominate its citation mix, which is consistent with Google's own public guidance that its AI features build on standard search indexing and ranking rather than any brand-submitted channel. The practical implication for marketers is significant: for Gemini, the visibility battle is fought on third-party ground. Being citable on your own domain matters for ChatGPT; being covered by sources Gemini already trusts matters for Google. Our team is currently building a third-party-coverage label so the next validation round can grade Gemini on the game it actually plays.

A validation pipeline that only ever produces good news is not a validation pipeline.

What do we refuse to claim from this data?

Honesty is the entire value of this exercise, so here are the limits, stated as plainly as the wins.

We do not claim causality for any individual factor. The regression's per-factor coefficients are not publishable as "do this, get cited." With 17 training brands, coefficients absorb confounds: famous brands get cited despite mediocre hygiene scores, so the model can learn perverse-looking weights that predict well but explain nothing. This is why we keep our shipped rubric weighted by controlled studies and platform documentation, not by this model.

We validated prediction across time, not across brand types. Later weeks of brands the model saw earlier is a real test, but it is not the same as scoring a brand category the model has never met. That test needs roughly double the brand diversity, and our corpus adds a new week of labels every week the scanners run.

The validated cohort is our previous audit engine. The current rubric inherits the same measurement core with evidence-based reweighting, and it accrues its own validation cohort from today forward. We will publish updated numbers as both cohorts grow.

Citations harvested through APIs proxy the consumer products. The overlap is high but not perfect, and AI answers churn: a citation earned this month can decay within a quarter. Weekly aggregation and repeated sampling blunt this, but no one measuring AI visibility should pretend the ground is still.

What should you do with this?

For brand owners, three actions follow directly from the data.

First, treat AI readiness as measurable, because it now demonstrably is: the factors our audit checks separate cited from uncited brands on unseen data, most strongly for ChatGPT, whose crawlers and search bot are documented and verifiable in your own server logs (OpenAI publishes its bot identities, and infrastructure providers have documented the rapid rise of AI crawler traffic).

Second, split your strategy by engine. ChatGPT rewards on-site readiness you control directly. Gemini rewards third-party coverage. A single "AI SEO checklist" that ignores this split is optimizing for an average that describes no engine.

Third, demand validation from any tool that scores you, including ours. As of July 2026, correlation studies are the industry ceiling. Ask one question: has this score ever been tested against observed citations it did not train on? As of this writing, we have published the only such test we know of, and we will keep publishing it as the numbers move, in either direction.

Frequently asked questions

What is AUC and why use it for AI visibility?

AUC (area under the ROC curve) is the probability that a model ranks a randomly chosen positive case above a randomly chosen negative one. 0.5 is coin-flip, 1.0 is perfect. It is the standard metric for asking "does this score separate the two groups" without committing to a single threshold, which fits citation prediction well because different brands care about different citation-rate targets.

Does an AUC of 0.84 mean the audit causes citations?

No. It means the audit's factor scores, taken together, rank brand-weeks by citation likelihood far better than chance on data the model never saw. Causal claims about individual factors need controlled experiments or far more brand diversity. We publish the discrimination result and withhold the causal one, because that is what the data supports.

Why is Gemini excluded from the results table?

Because in our corpus Gemini almost never cites any brand's own domain, there are too few positive cases to grade a predictor honestly. Excluding it with the reason stated beats padding the table. Its answers cite third-party sources instead, which requires a different outcome label we are now building.

Will these numbers change?

Yes, and they should. The corpus grows by every scan week, the current rubric is accruing its own validation cohort, and citation behavior itself drifts as engines update. We treat this as a standing benchmark, not a one-time trophy, and the methodology page will always carry the latest run, updated as the corpus grows.

Sources