---
title: Can a Content Score Predict Whether AI Will Cite You?
description: A new validation study re-measured the classic GEO effect sizes on ten modern engines and found they move citation on none. What an engine-free content score can and cannot tell you.
canonical: https://www.aeonav.fivecoder.com/blog/can-content-scores-predict-ai-citations
updated: "2026-09-11"
---

# Can a Content Score Predict Whether AI Will Cite You? (/blog/can-content-scores-predict-ai-citations)

**Every "AI citability grade" rests on the same 2023 experiment, and that experiment just failed to replicate.** A [preprint posted 7 September 2026](https://arxiv.org/abs/2609.07559) by Elisha Bajemon and Andre-Louis Rochet set out to validate a deterministic content score — the engine-free kind that audit tools and dashboards hand out as a number from 0 to 100 — and ran the field's only published causal effect sizes against ten modern engine families along the way. The three strongest interventions from the 2023 GEO study — add quotations, add statistics, cite sources — moved citation on none of them.

**Key facts**

| 2023 GEO intervention | 2023 effect (visibility metric) | July 2026 re-measurement, 10 engine families |
|---|---|---|
| Quotation addition | +42.6 | **No measurable effect on citation** |
| Statistics addition | +32.8 | **No measurable effect on citation** |
| Cite sources | +27.7 | **No measurable effect on citation** |
| Keyword stuffing | −8.8 | Still null-to-negative |

*Source: [Bajemon and Rochet, arXiv:2609.07559](https://arxiv.org/abs/2609.07559), 7 September 2026, re-running the [KDD 2024 GEO](https://arxiv.org/abs/2311.09735) interventions on six open-weights and four commercial engine families.*

| | |
|---|---|
| Content-only prediction vs actual citation | **Spearman ρ ≈ 0.11** within-query — roughly 1% of the variance |
| Overfitting: in-sample → out-of-sample | **0.90 → 0.20**, 11 weights fitted on fewer than 40 observations |
| Maximum score gain from stuffing any single lever | **+6.1 of 100 points**, on a 500-source adversarial benchmark |
| Independent corroboration | [C-SEO Bench](https://arxiv.org/abs/2506.11097) (NeurIPS 2025): most conversational-SEO methods ineffective or negative |

**Key takeaways**

- The 2023 anchors are expired. Quotation addition measured +42.6, statistics +32.8, and cite-sources +27.7 in the original GEO experiments; the July 2026 re-measurement on six open-weights and four commercial engine families, including three gpt-5.x arms, found effects "statistically indistinguishable from zero where not nominally negative."
- Content alone barely predicts citation. The study's best content-only predictor — eleven deterministic text features — correlated with actual within-query citation at Spearman ρ ≈ 0.11.
- Fitting a score to observed citations overfits at realistic sample sizes: Spearman 0.90 in-sample collapsed to 0.20 out-of-sample, with eleven weights fitted on fewer than 40 noisy observations.
- What survives validation is a quality filter, not a crystal ball: a score whose gaming is provably bounded (max +6.1 of 100 points from lever-stuffing) is useful for catching padding and duplication, and useless for forecasting visibility.
- This is the third independent result pointing the same way: C-SEO Bench (NeurIPS 2025) found most conversational-SEO methods ineffective or negative, and the competition simulations we covered [in August](/blog/do-geo-tactics-stop-working) showed fixed tactics inverting under adoption. The disagreement between 2023 and 2026 has explanations; the direction does not change with the explanation.
- The practical inversion: an engine-free grade cannot substitute for measuring answers, because the thing it omits — retrieval, at ask time — is where most of the outcome lives. Our own tracking put day-to-day churn in cited domains at [60–70%](/blog/citation-volatility).

**On this page:** [Why these scores exist](#why-do-ai-content-scores-exist) · [What the paper did](#what-did-the-paper-do) · [Reconciling 2023 and 2026](#why-do-the-2023-and-2026-geo-results-disagree) · [Which grades this affects](#which-ai-content-scores-does-this-affect) · [What a score is still for](#what-is-a-content-score-still-good-for) · [Limits](#what-this-does-not-show) · [What to do](#what-to-do-on-monday)

## Why do AI content scores exist? [#why-do-ai-content-scores-exist]

The appeal of a content score is obvious once you price the alternative. The ground truth for AI visibility — run real prompts against real engines, record who gets cited — is expensive, rate-limited, and unstable from day to day. A number computed from the page itself is free, instant, and reproducible. So the market has produced them in volume: audit tools that grade "AI readiness," dashboards that score pages for citability, linters that check Markdown against answer-engine heuristics. Behind almost all of them sits the same intellectual collateral: the [KDD 2024 GEO paper](https://arxiv.org/abs/2311.09735), which reported that content edits could boost generative-engine visibility "by up to 40%," with quotations, statistics, and cited sources as the strongest levers.

The new preprint asks the uncomfortable prior question: how would you know whether such a score measures anything? Its answer is a validation protocol — and the protocol's first casualty is the collateral.

## What did the paper do? [#what-did-the-paper-do]

The authors build the strongest honest version of the thing being sold: a deterministic score with eleven sub-components (information density, quotable density, entity density, statistic density, self-containment, semantic redundancy, and the like), all computable with regexes, counting, and TF-IDF — no model call, no engine. Then they subject it to two kinds of check.

The first is a battery of falsification gates: a negative control that must not move the score, a dose–response requirement, a proof that amplifying any single lever gains a bounded number of points, a duplication penalty, and length neutrality. On a 500-source adversarial benchmark built from GEO-Bench, the worst attacker gain was 6.1 points on the 0–100 scale — quotation-stuffing at the lowest dose — and gains shrank as the dose rose. These gates are the paper's genuine contribution: they define what the score is robust to, and almost no commercial grade has been tested against any of them.

The second check is external: does the score's construction agree with measured causal effects on real engines? The only published effects are the 2023 ones, so the authors re-ran those interventions — same edits, modern engines — in July 2026, across ten engine families. The result is the sentence the whole field should read twice: the strongest 2023-era GEO interventions "move citation on none of ten engine families." The paper demotes the 2023 numbers to "an expired external check." And recalibrating the score to the honestly measured modern effects has a telling consequence: it zeroes out the lever-responsive features entirely. A 2026-honest content score contains no GEO-tactic levers, because the levers no longer carry measurable weight.

One more number completes the picture. Even allowing the score every advantage — conditioning on the query, using all eleven features — the correlation between the content-only prediction and which sources engines actually cited was a within-query Spearman of about 0.11 — meaning content features account for roughly 1% of the variance in which sources get cited. Not zero: better text is weakly favored. But at ρ ≈ 0.11 the overwhelming share of the citation outcome is decided by something the page's text does not contain.

## Why do the 2023 and 2026 GEO results disagree? [#why-do-the-2023-and-2026-geo-results-disagree]

Read in isolation, one preprint reporting null effects would deserve suspicion. Read in sequence, it is the third point on a line.

### 2023 — the levers work [#2023--the-levers-work]

The original GEO experiments measure large effects: up to 40% visibility gains, quotations and statistics leading. But the engine is a research sandbox built on 2023-era models, the metric is position-adjusted word share of the answer, and — as the [competition literature](/blog/do-geo-tactics-stop-working) later formalized — one document is optimized while every competitor stands still.

### 2025 — the levers mostly don't [#2025--the-levers-mostly-dont]

[C-SEO Bench](https://arxiv.org/abs/2506.11097), by Haritz Puerto and colleagues (NeurIPS Datasets & Benchmarks 2025), tests ten conversational-SEO methods — including the same quotations, statistics, and citations — on GPT-4o-mini, Claude 3.5 Haiku, o3, and o4-mini. Most are "largely ineffective" and frequently negative; the only methods with modest positive effects are holistic content improvement and instruction-like LLM guidance, and the paper's conclusion is that traditional retrieval-rank SEO "remains essential" — if the engine's search does not fetch your page, no phrasing rescues it.

### 2026 — the levers measure zero [#2026--the-levers-measure-zero]

The Bajemon–Rochet re-measurement closes the loop on modern commercial engines, including gpt-5.x arms.

The disagreement has candidate explanations, and they matter because they point at different failure modes. The engines changed: grounding, reranking, and answer synthesis in 2026 bear little resemblance to a 2023 sandbox, and the paper's own framing — the engine as a "non-stationary" oracle — implies every effect size carries a timestamp. The metrics differ: word-share of an answer is not the same outcome as being cited at all. And adoption eroded whatever was real: the [Capital One simulations](/blog/do-geo-tactics-stop-working) showed fixed tactics inverting as the optimized share of the pool rises, and the [CISPA prevalence measurement](/blog/geo-detection) shows that share climbing year over year. The three explanations are not exclusive, and none of them rescues the practitioner claim — that adding quotation blocks and statistics to a page reliably raises its citation rate on today's engines. No current measurement supports that claim, and the only current measurements we have contradict it.

What the reconciliation does *not* support is the opposite overcorrection. Retrieval still decides most of the outcome, so the boring machinery that gets pages retrieved — crawlability, [structured, extractable content](/blog/structured-data-for-ai-search), topical authority, being the page a search returns — is precisely what C-SEO Bench found still matters, and what [Google's own AI optimization guidance](https://developers.google.com/search/docs/fundamentals/ai-optimization-guide) describes as the whole of the job. What expired is the idea that a cosmetic layer of GEO edits on top, or a grade measuring that layer, moves citation.

## Which AI content scores does this affect? [#which-ai-content-scores-does-this-affect]

Any grade whose construction rewards the 2023 levers — and that is most of them, because the 2023 paper is the only published causal evidence the category ever had. The surface area is wide: AI-readiness audits, per-page citability grades, "AEO scores" bolted onto existing SEO scores, free graders used as lead magnets, and internal models a team fits on its own citation history. The [AI visibility tool directory](/ai-visibility-tools) lists which platforms ship one.

Rather than take any vendor's word for it, put three questions to every grade in your stack:

| Question | What a good answer sounds like | What should worry you |
|---|---|---|
| Validated against what outcome? | "Against citation rates we measured on live engines, and here is the correlation" | "Against best practices" — a restatement of the construction, not a test of it |
| On which engines, and when? | Named engines, named model versions, a measurement date | No date, or engines unnamed. Effect sizes have timestamps; these expired in under three years |
| Can I see the falsification tests? | Negative controls, dose–response, a bounded gain from stuffing any single lever | Nothing. Almost no commercial grade has been tested against any of the paper's gates |

The self-built version deserves the same scrutiny. Fitting eleven weights on forty noisy observations scored **Spearman 0.90 on the data it saw and 0.20 on data it did not** — so an internal "citability model" trained on a few weeks of your own citations is running the same trap at the same sample size.

## What is a content score still good for? [#what-is-a-content-score-still-good-for]

The paper does not conclude that content scores are worthless — it concludes they are mislabeled. Stripped of expired causal claims, what survives the gates is a quality filter: a cheap, deterministic, manipulation-bounded check for duplication, padding, incoherence, and thin rewrites, positioned behind web-spam detection (which, the authors note, dominates it for catching actual attacks). Their recommended uses are corpus filtering, editorial quality control, and leaderboards — a linter, in software terms. A linter is genuinely useful. Nobody mistakes a clean lint run for a forecast that users will love the program.

That reframing gives an in-house team a clean rule for every grade a tool shows them: read a low score as a defect list and a high score as the absence of known defects. The moment a grade is read as *predicted visibility* — "we moved from 71 to 88, so citations should follow" — it is being read as exactly the thing that failed validation.

## What this does not show [#what-this-does-not-show]

The usual discipline applies. This is one preprint, and its re-measurement runs on GEO-Bench sources — benchmark documents, not your product pages — with "citation" defined by the benchmark's protocol, not by a production Perplexity or ChatGPT session. Ten engine families is broad for an academic study and narrow against the surface area of deployed answer engines. The within-query 0.11 is a correlation under one feature set; a richer representation might extract more signal from content alone, though the direction of every recent result suggests not much more. And a null effect on citation does not rule out subtler effects the protocol cannot see — on how a brand is described once cited, for instance, which is a different outcome from whether it is cited at all.

The overfitting warning cuts in all directions, including toward do-it-yourself dashboards: eleven weights fitted on forty observations scored 0.90 on the data it saw and 0.20 on data it did not. Any internal "citability model" trained on a few weeks of your own citations is running the same trap at the same sample sizes.

## What to do on Monday [#what-to-do-on-monday]

1. **Inventory the grades you already pay attention to.** Every "AI readiness" or citability number in your stack gets the three vendor questions: validated against what outcome, on which engines, measured when. Anything anchored to the 2023 effect sizes is anchored to evidence that failed replication in July 2026.
2. **Reclassify grades as lint.** Keep them in the editorial workflow for what they catch — duplication, padding, missing structure. Remove them from any dashboard where they sit next to visibility metrics as if they predicted them.
3. **Stop buying points.** A content sprint whose success criterion is a higher grade is optimizing a proxy with a measured ceiling of about 6 points and a measured predictive value of about 0.11.
4. **Spend the difference on retrieval and measurement.** The retrieval layer — whether engines fetch your pages at all — is where the outcome is decided, and the measurement layer is the only validation that transfers to your category: join your pages' grades against their actual citation rates in [your tracked prompts](/blog/ai-prompt-tracking), and believe your own join over any benchmark.
5. **Timestamp everything.** The single deepest point in the paper is that the oracle is non-stationary. Whatever relationship you establish between content features and citations this quarter is a fact about this quarter's engines.

The honest summary: the industry built scoring products on a causal result faster than anyone checked whether the result aged, and it did not age. What replaces the grade is not a better grade — it is measuring the outcome the grade was standing in for.

[Aeonav](https://aeonav.fivecoder.com/) is an open-source, self-hosted AI visibility platform that measures the oracle directly: it runs your prompt set on a schedule across ChatGPT, Claude, Gemini, Perplexity, and Google's AI surfaces and records every answer, mention, and cited page in your own database. That makes the validation this research calls for a query — join any content score against the pages engines actually cite for your prompts, and find out what predicts citation *in your category* instead of on a benchmark.

For the fundamentals, start with [what AI citations are](/blog/ai-citations). For the adjacent evidence, see [whether GEO tactics survive competition](/blog/do-geo-tactics-stop-working), [how much of the web is already optimized](/blog/geo-detection), and [how fast cited sources churn](/blog/citation-volatility). For the practice, the [AEO guide](/answer-engine-optimization) and [how to track your brand in AI search](/answer-engine-optimization). For the vocabulary, the [AI search glossary](/glossary).