This post describes a preprint that has not yet been peer reviewed. All findings are observational associations, not causal claims or tested optimization strategies.

When you ask ChatGPT a question with web search on, it does three things that a classic search engine does not fully expose: it rewrites your question into one or more search queries, it pulls back a pool of candidate pages, and then it chooses which of those pages to show you as citations. Only the last step is visible to the user. For anyone who cares about being cited by a generative engine, whether a brand, a publisher, or a researcher, that hidden middle is exactly where the action is. This work on generative engine optimization (GEO) continues the thread started with GEM-Bench and VoteGEO-Bench, but turns the camera around: instead of measuring the answer, we measure the process that produces it.

What we measured, and how

We sampled 80 queries from Search Arena, a crowd-sourced dataset of search-augmented LLM conversations, balanced across four intents (factual lookup, information synthesis, analysis, recommendation) and two languages (Chinese and English). Each query was sent repeatedly to four production settings of ChatGPT Web between late March and late July 2026: 5.2 Instant, 5.3 Instant, 5.4 Thinking, and 5.5 Instant, for roughly 4,200 runs in total. From the server-sent event traces of each run we extracted the executed search queries, the retrieved candidate pages, and the final citations.

Because exact URLs are a very strict way to compare sources, every analysis is done at three levels: the URL, the hostname (so a subdomain counts as its own source), and the entity (brands, products, and other named entities extracted from page titles and snippets). This turns out to matter a lot.

Finding 1: The settings behave differently before any page is retrieved

The Instant settings are mostly single-round: one search, one pool, one answer. But they differ in how they rewrite the question. 5.2 Instant issued more than two queries per run and, for Chinese questions, almost always searched in both Chinese and English. 5.3 and 5.5 Instant mostly issued a single English query. 5.4 Thinking is a different animal: it averaged nearly 12 executed queries over about three retrieval rounds, and more than 70% of its runs used search more than once. Later rounds became narrower, with far more site: operators and quoted phrases. Whether those extra rounds improve the citations is a question we deliberately do not answer here; the point is that "what ChatGPT searched for" depends on the setting, not just on what the user typed.

Finding 2: Retrieval is noisy, but recurrence predicts citation

Run the same query twenty times and you get twenty noticeably different candidate pools. At the URL level, roughly 40% of candidates appeared in only one run, and the stable core that showed up in most runs was under 10%. Hostnames and entities were steadier. The useful part is what happens next: a candidate that appears often across the other runs is more likely to be cited in a held-out run, with leave-one-run-out AUCs rising from about 0.62 at the URL level to about 0.69 to 0.71 at the entity level. Snippet length was a weaker but consistent signal, retrieval position mattered without being decisive (candidates ranked beyond fifteenth were still cited around a fifth of the time), and recency barely registered.

One caution the paper makes explicitly: this does not mean that publishing more pages or forcing a source to appear more often in retrieval will earn more citations. Recurrence is an observable correlate of what the engine finds useful, not a lever we tested.

Finding 3: External proxies see only part of the picture

Most GEO monitoring cannot read ChatGPT's internal traces. It relies on proxies such as Google Search results or the OpenAI Responses API with web search enabled. We asked how well those proxies recover what ChatGPT Web actually cited. Responses API candidates gave the broadest coverage and its citations matched ChatGPT Web's most closely, while Google alone recovered under 10% of cited URLs. Even combined, the two proxies recovered only about a quarter of cited URLs (28.3% and 26.2% for the 5.3 and 5.5 Instant settings), but about half of cited hostnames, and a similar share of cited entities. If your monitoring works at the page level, it is missing most of what gets cited. If it works at the site or brand level, it is doing considerably better.

Finding 4: The external signals are still worth combining

Within the external candidate set, we trained a gradient-boosted model on signals that any outside observer can compute: Google and Responses API rank, how often a candidate was exposed and how often it converted to a citation in the proxy runs, whether multiple URLs supported the same hostname or entity, and snippet length. The result reached AUCs of 0.78 at the URL level, 0.76 at the hostname level, and 0.74 at the entity level for both the 5.3 and 5.5 Instant settings, beating a baseline that simply trusts Responses API citations by 0.07 to 0.11 AUC. The mix of informative signals shifted between settings: rank dominated for 5.3, while exposure and conversion grew in importance for 5.5. These AUCs are conditional on the proxy candidate set, so the coverage ceiling from Finding 3 still applies end to end.

What this means for GEO

• Separate retrieval visibility from citation visibility. Appearing in search results is a precondition for citation, not a guarantee. Search rank should not be the final objective.
• Monitor executed queries, not just user queries. Rewriting differs across settings, including the language the engine searches in.
• Use repeated runs, and check which model actually served each one. A single response is an incomplete sample. During collection, two sessions silently switched to fallback models after a dozen or so successful runs.
• Track hostnames and entities alongside exact URLs. They are more stable across runs and better covered by external tools.
• Re-evaluate when the deployed setting changes. Rewriting behavior, proxy effectiveness, and predictive signals all shifted between settings, and the study does not isolate the effect of any single model update.

Limitations

This is an observational study of one engine. Two of the four settings were deprecated mid-collection, so their run counts are smaller and the 5.4 Thinking query set is not matched to the Instant sets. Feature importances describe associations, not causes. The entity layer mixes brands with other named entities, so the results do not support separate conclusions for brands. Extending the same lens to other generative engines is the natural next step.