TL;DR
Conversational Search Engine Optimization (C-SEO)—the practice of tweaking web text so that AI search engines cite it first—is largely ineffective and often hurts citation rankings. Across a new multi-domain benchmark, traditional SEO (ranking high in the initial retriever so text appears early in the LLM’s context window) remains far more influential, while competitive C-SEO adoption quickly degrades into a congested, zero-sum game.
What Problem Does This Paper Address?
As search engines transition from ranked lists of links to Conversational Search Engines (CSEs) that generate synthesized answers with inline citations (e.g., Perplexity, ChatGPT Search, Google AI Overviews), content providers want to know how to optimize for visibility.
Early literature suggested that "white-hat" C-SEO—stylistic edits like making text more authoritative, adding statistics, or improving fluency—could boost an article’s presence in generated answers. However, prior studies evaluated narrow domains, relied on single-actor assumptions (assuming no competitors optimize their text), and evaluated visibility using raw word count rather than true citation priority. This paper investigates whether white-hat C-SEO genuinely helps content get cited earlier, how it performs across varied tasks and domains, and how competitive adoption affects performance.
Methods
The authors evaluate C-SEO by formalizing the objective: minimizing the citation rank ($\min \text{rank}(c_i)$) so that an LLM cites the target document earlier in its generated response.
The study tests ten white-hat transformation methods:
- Eight stylistic transformations from prior work: Authoritative, Statistics, Citations, Fluency, Unique Words, Technical Terms, Simple Language, and Quotes.
- Two newly proposed methods:
- Content Improvement: A holistic rewrite combining all eight stylistic transformations into structured, polished text.
- LLM Guidance: Prepending a markdown summary to the document, inspired by the
llms.txtweb standard.
Experiments were conducted using gpt-4o-mini to apply transformations, and evaluated on four frontier engines: gpt-4o-mini, claude-3-5-haiku, o3, and o4-mini. The authors also modeled competition by varying the adoption rate ($\alpha \in [0, 1]$), calculating the Area Under the Curve (AUC) across adoption levels to capture game-theoretic dynamics.
Datasets
The authors introduce C-SEO Bench, comprising over 1,900 queries and 16,300 documents across two primary tasks:
- Product Recommendation (10 items retrieved, top 5 recommended):
- Retail: 500 queries from the Amazon Shopping Queries dataset.
- Video Games: 436 queries using Steam store tags and game descriptions.
- Books: 249 genre/mood queries mapped to Google Books API entries.
- Question Answering (5–8 candidate passages):
- Web Questions: 300 queries from Natural Questions with web pages retrieved via Brave Search.
- News: 294 events from the Multi-News dataset.
- Debate: 142 controversial query snippets with supporting/opposing arguments.
Key Insights and Ideas
- C-SEO is mostly ineffective—and often harmful: For most combinations of tasks, domains, and models, stylistic modifications did not produce statistically significant rank gains. In fact, aggressive strategies like adding arbitrary statistics frequently triggered significant rank demotions.
- Context position (Traditional SEO) dominates generation: The order in which retrieved documents enter the LLM prompt (retrieval order) heavily dictates citation priority. Simply ranking first in the retrieval context yielded rank boosts up to an order of magnitude higher than any content modification. C-SEO cannot substitute for ranking well in traditional retrievers.
- A congested, zero-sum game: As more competitors adopt C-SEO techniques, individual advantages dilute rapidly toward zero. Early adopters gain minimal margins that disappear as market adoption saturates.
- Word count misled earlier findings: Prior claims of C-SEO success measured how many words the LLM used when discussing a source. However, more verbose coverage does not mean the LLM prioritized that document over competitors in citation order.
Key Results
- Rare positive boosts: Out of 54 evaluated settings on
gpt-4o-mini, only 3 showed statistically significant rank improvements: LLM Guidance on Retail (+0.36) and Video Games (+0.24), and Content Improvement on Retail (+0.18). - Zero significant wins on other models: On Claude 3.5 Haiku, o3, and o4-mini, no C-SEO method achieved statistically significant gains.
- Frequent performance penalties: The Statistics method significantly decreased citation rankings in 19 out of 24 settings. In product recommendations on Haiku 3.5, 26 out of 30 settings had statistically significant negative effects.
- Traditional SEO outclasses C-SEO: Placing a document first in the LLM context window (Best SEO) provided massive, statistically significant boosts across all domains (e.g., +2.77 positions in Retail, +1.89 in Games, +1.54 in Debate on
gpt-4o-mini), completely dwarfing the best C-SEO gains (+0.36 max).
Limitations and Future Work
- Exclusively white-hat: The study does not evaluate adversarial black-hat methods (such as prompt injections or jailbreaks aimed at actively suppressing competitors).
- Context fit assumptions: All retrieved documents were short enough to fit inside the LLM context window; truncation or extreme long-context retrieval dynamics were not evaluated.
- Linguistic and model scope: The benchmark is limited to English-language content and closed, commercial LLMs.
- Future directions: Exploring potential synergies between SEO and C-SEO methods, testing open-weight models, and analyzing how intermediate document summarizers alter citation order.
Summary updated