Retrieved but Not Cited: The Real AI Citation Gap

Check your server logs right now and there's a decent chance you'll find GPTBot, ClaudeBot, and PerplexityBot all over them — crawling your service pages, your FAQ, your blog, on a schedule that looks a lot like real interest. Then ask the actual AI platform a question your page answers. Nothing. No mention, no link, no citation. Your content got read. It just didn't get used. That specific, quietly infuriating gap has a name now, and in September 2026 it's finally getting measured properly instead of just complained about.

A researcher who runs a small AI-visibility tool put a number on it that landed hard on Hacker News: after auditing hundreds of real business websites, only 8.9% of sites block AI crawlers in robots.txt, but 94.8% of them are never named in an actual AI answer. Almost nobody is opting out of being read. Almost everybody is being ignored anyway. That's not a content problem in the way most businesses assume — it's a retrieval-versus-citation problem, and the two are measurably, structurally different things.

Two Different Games, Not One

Retrieval is the easy part now. Modern AI systems pull in dozens of candidate pages for a single query, sometimes far more than what shows up in the final answer — that's just how retrieval-augmented generation works. Citation is a separate, much harder filter applied on top of that pool, and most pages that clear the first bar never clear the second. A March 2026 analysis by ALM Corp, working across 548,534 pages retrieved by ChatGPT across 15,000 prompts (43,233 queries once you count the fan-out expansions ChatGPT silently runs behind a single question), found that only 15% of retrieved pages were ultimately cited in the final response. Eighty-five percent got pulled in, evaluated, and dropped. Peec AI's platform-level data tells the same story from a different angle. Looking at over a million tracked citations, they found 64% of URLs tracked on Perplexity never receive a single citation, even though Perplexity's interface shows those same sources sitting right there in its sidebar as retrieved material. The page is visibly present. It's just not the one that gets named.

Here's why this distinction actually matters for what you do next: if you're only measuring whether your page shows up in search-adjacent tools, or whether your server logs show bot traffic, you're measuring retrieval — the part that's already mostly solved for you, since these bots crawl the open web fairly indiscriminately. Citation is the part that requires actual work, and it's the part almost nobody separates out when they report on "AI visibility."

A Real Framework for Measuring the Gap

Until recently, most of what got written about this was anecdotal — a screenshot of a citation list, a complaint thread, a vague sense that something was off. That changed with a genuinely rigorous piece of research published in April 2026 by three independent researchers, Zhang Kai, He Xinyue, and Yao Jingang, titled "From Citation Selection to Citation Absorption: A Measurement Framework for Generative Engine Optimization Across AI Search Platforms." It's built on a dataset of 602 prompts and 21,143 valid citations across ChatGPT, Google AI Overviews/Gemini, and Perplexity, and it does something most industry blog posts don't: it splits "getting cited" into two genuinely separate, separately measurable stages.

The first stage, which they call citation selection, is exactly what it sounds like — does the platform pull your page in and list it as a source at all. The second, citation absorption, asks something sharper: once you're on that list, does the answer actually reflect your content, or are you a name tacked onto a paragraph the model wrote from somewhere else entirely? They built an influence score to quantify absorption, combining reference frequency, where in the answer your citation lands, how much of the answer's paragraph coverage traces back to you, TF-IDF semantic similarity between your page and the generated text, and — the part that maps directly onto what "genuine influence" actually means — bigram and trigram overlap between your actual wording and the model's actual output.

The headline finding is the one worth sitting with: ChatGPT cited fewer sources per prompt than Google or Perplexity (6.88 on average, versus 12.06 and 16.35), but the sources it did cite showed 4.6 times higher absorption scores. In other words, being cited more often and being cited more meaningfully are not the same axis, and a platform that names you constantly might still be treating your content as decoration rather than substance. The researchers also found news sources get cited frequently — 16 to 31% of all citations across the three platforms — but absorbed weakly, with a mean influence score of 0.0726, against 0.2144 for encyclopedic sources. Getting cited a lot and getting used are, again, two different outcomes.

What Makes a Retrieved Page Lose the Citation

So if retrieval is close to guaranteed and citation isn't, what actually separates the pages that make the cut? The ALM Corp dataset gives some genuinely specific answers. Pages with 50% or more word overlap between their title and the actual query got cited at a 20.1% rate, against 9.3% for pages with under 10% overlap — a 2.2x gap that has nothing to do with domain authority and everything to do with how precisely the page's own language matches what was actually asked. Freshness mattered almost as much: content updated within the past three months was roughly twice as likely to be cited as older content, even when the older content ranked fine on Google. And structure decided where the citation actually pointed — 44.2% of citations drew from content sitting in the first 30% of a page, meaning a correct, well-sourced answer buried under three paragraphs of preamble is functionally invisible to the part of the pipeline doing the selecting.

The Zhang/He/Yao paper adds a structural layer to that same story. Pages in the top quartile for citation absorption averaged 1,943 words, against 170 for the bottom quartile — an 11.4x gap — and high-influence pages carried roughly 12.5 times more headings and 8.9 times denser lists than low-influence ones. Certain content types performed distinctly better as source material too: pages built around definitions saw a 57% absorption boost, comparisons 55%, code examples 77% — while, in a finding worth remembering before restructuring everything into Q&A blocks, plain question-and-answer formatted pages actually underperformed by about 5.74% relative to non-Q&A pages. The researchers describe the winning pattern as an "evidence-container" — a page that reads less like a persuasive essay and more like a modular set of clearly labeled, independently extractable facts a model can lift a piece from without needing the surrounding argument to make sense.

None of that is about being wordier or keyword-stuffing a headline. Ahrefs' own analysis of AI Overviews citations has previously shown domain authority doesn't cleanly predict citation share the way it predicts traditional rank — this newer research explains part of why: the model isn't rewarding your site's reputation, it's rewarding whether your specific sentence is the cleanest, most literally matching answer sitting in the retrieval pool at that moment.

The Deeper Problem: Citations That Don't Even Reflect the Source

There's an uncomfortable layer underneath all of this that predates the "AEO" conversation entirely. Back in 2023, Stanford researchers Nelson Liu, Tianyi Zhang, and Percy Liang ran a careful human-audit study, "Evaluating Verifiability in Generative Search Engines," published at EMNLP, testing four systems (Bing Chat, NeevaAI, Perplexity, and YouChat) against real queries. They found only 51.5% of generated sentences were fully supported by the citations attached to them, and only 74.5% of citations actually supported the sentence they were attached to. That means roughly a quarter of the time, the little footnote number next to a claim doesn't actually back up the claim — the citation is present, but it's decorative.

Put that together with the retrieval-vs-selection-vs-absorption chain and you get the full, unflattering picture: your page can be retrieved, evaluated, correctly cited, and still misrepresent what it actually says, because the system generated the sentence first and attached your link afterward as justification rather than building the sentence from your actual wording. That's exactly the mechanism the influence score in the Zhang/He/Yao paper is trying to catch — a citation with near-zero phrase overlap to the source it's attributed to is a citation in name only.

"Are We Saying It's Now a Problem That We're Not Getting Scraped?"

Worth reading the actual argument people have about this rather than only a summary of it, because it's more contested than the headline stat suggests. The Hacker News thread discussing that 8.9%-versus-94.8% study pulled in 61 comments, and the disagreement in it is genuinely useful. The author, posting as SpikeyCoder, framed the core issue in a comment that's become something of a reference point in these discussions: "In the AI Search era, businesses are being scraped to build these answers, but they have zero telemetry on whether they are actually making the cut." Small businesses, he added later in the thread, "don't [have that visibility]. But somehow those larger companies seem to get cited."

"How LLMs cite is entirely opaque."

— SpikeyCoder, replying in the Hacker News thread on AI citation rates

Not everyone agreed it's actually a problem. A commenter posting as brookst pushed back hard: "This is a weird complaint... Are LLMs supposed to link to every site? You could say the same thing about the percent of sites crawled by Google versus ever showing up on first page of results." Fair point — nobody expected 100% of the indexed web to rank on page one either. But another commenter, ButlerianJihad, raised a sharper technical objection that cuts closer to the verifiability research above: "It is architecturally impossible for an LLM to associate a link or citation that it crawled with a response... Every link they're giving you to support their statements is tacked on." That claim didn't go unchallenged — freedomben replied that "increasingly there are a lot of LLMs that are essentially doing RAG against search results," meaning the newer generation of systems is architecturally closer to actually grounding citations in retrieved text rather than bolting links on after the fact, even if plenty of older or simpler implementations still work the sloppier way. The honest read of that whole exchange is that both things are true depending on which system and which mode you're actually testing — which is precisely why measuring your own specific answers on your own specific platforms beats assuming any general rule applies to you.

Why This Hits Harder in a Bilingual Market

For a business operating across the UAE and wider Gulf, the retrieval-versus-citation gap doesn't split cleanly by language the way you might expect — it can actually be worse on the Arabic side, precisely because Arabic content pools are thinner. If ALM Corp's title-overlap finding holds — that AI systems reward the literal, specific match between what was asked and what a page says — then a business whose Arabic pages are a loose or partial translation of the English original is handing the model weaker phrase-level material to absorb from, even when the underlying facts are identical. Fewer competing Arabic sources also means fewer chances for a model to find a well-structured, freshly dated, specifically worded Arabic answer anywhere — which cuts both ways: it's a real gap to close, and it's also a genuine opening for whoever closes it first. Checking whether your Arabic content clears the citation bar, not just the retrieval bar, means testing it separately rather than assuming parity with your English results.

What Actually Measuring This Looks Like

Put the research together and the practical shape of the work is fairly specific, not vague. It starts with establishing whether you're even in the retrieval pool for the questions that matter to your business — genuinely likely, given how indiscriminately these crawlers work. Then it moves to the harder question: when you are retrieved, are you cited, and separately again, does the generated answer show real phrase-level and structural overlap with what your page actually says, or is your name sitting next to a sentence someone else's content effectively wrote? That three-stage distinction — retrieved, cited, actually reflected — is the whole point of measuring citation contribution rather than citation presence, and it's the part most AEO vendors skip because presence is the easy number to report and contribution requires the kind of content-level comparison the Zhang/He/Yao influence score and the Stanford verifiability study both had to build custom methodology to even attempt.

That's the specific gap our own AI Citation Optimization work is built to close — not just checking whether your business shows up in a citation list, but comparing the actual generated answer against your actual published content to see whether real influence happened or whether the citation is cosmetic. If you want a free first read on where your own content currently stands before committing to anything, the AEO Score tool gives you a real, evidenced starting point in a few minutes.

Where This Leaves You

The uncomfortable truth in all of this research is that being crawled was never the finish line — it was barely the starting gate. Your logs showing GPTBot traffic tell you a model looked at your page. They tell you nothing about whether it believed your page, used your page's actual words, or quietly cited a competitor's cleaner, fresher, more specifically worded answer to the exact same question instead. The gap between those two outcomes is now something you can actually measure rather than just suspect from an unexplained traffic spike with nothing to show for it. Given how volatile this all is — the same Hacker News thread that produced 61 comments of genuine disagreement is itself proof the field hasn't settled — the businesses that check this regularly, on their own real questions, in their own real language, are the ones who'll notice the gap closing instead of guessing whether it ever did.