TL;DR
Generative search engines (GSEs) exhibit a significant "rich-get-richer" bias, disproportionately recommending market-dominant brands over niche or new competitors. This favoritism is deeply embedded in the models' internal memory and persists even when they search the live web, creating a "parametric barrier to entry" that standard content optimization cannot easily overcome.
What Problem Does This Paper Address?
As we move from traditional search engines (lists of links) to generative search (synthesized conversational answers), the AI becomes a powerful gatekeeper. If you ask a chatbot for the "best running shoes," it doesn't just give you a list; it picks a few winners and justifies them.
The authors investigate whether these recommendations are meritocratic or biased toward fame. They define two types of bias:
- Brand Popularity Bias: A systematic preference for high-market-share brands over the "long-tail" of smaller competitors.
- Brand Familiarity Bias: The ranking advantage a brand gets simply because it was seen frequently during the LLM’s pre-training, compared to a brand the model doesn't "know."
This matters because if AI search engines only recommend what is already popular, they risk creating digital oligopolies, stifling competition, and hiding superior but less-famous products from consumers.
Methods
The researchers conducted two distinct studies to isolate these biases:
- Study 1 (The Audit): An observational study using 80 commercial prompts across four sectors (Sport, Fashion & Beauty, Insurance, and Travel). They tested three major engines—GPT-4o (referred to as gpt-5.5 in the paper's internal notation), Gemini-1.5-Flash, and Perplexity—under both "search-on" (Retrieval-Augmented Generation) and "search-off" (parametric memory only) conditions.
- Study 2 (The Experiment): A controlled "lab" experiment where they held product content constant but swapped the brand signatures. They created five versions of product pages: the original brand, a more popular real brand, a less popular real brand, and an invented brand. They also applied Generative Engine Optimization (GEO) techniques—rewriting text for better structure, fluency, or authority—to see if better writing could beat brand recognition.
Datasets
The study utilized several data sources to build its benchmarks:
- Keyword Data: Sourced from DataForSEO to identify real-world commercial queries in the Italian market.
- Brand Popularity: Measured using mean monthly Google search volumes in Italy for 1,176 top market players.
- Product Pages: Real product descriptions were scraped from official brand websites to serve as the baseline for the controlled experiments in Study 2.
Key Insights and Ideas
- The Parametric Barrier: The bias isn't just in the search results; it's in the model's "brain." Even when search is turned off, LLMs have a strong internal preference for popular brands.
- Search Doesn't Fix It: You might expect live web search (RAG) to level the playing field by finding niche brands. Instead, the authors found that the models often write popular brands into their initial search queries before they even see the web results.
- Identity Over Content: In the controlled experiments, swapping a real brand name for an invented one (while keeping every other word of the product description identical) significantly tanked the product's rank.
- GEO is Not a Silver Bullet: While "Generative Engine Optimization" is a growing field, this paper found that fact-preserving rewrites (making text more "authoritative" or "fluent") provided negligible ranking gains compared to the massive boost provided by a recognizable brand name.
Key Results
- Popularity Multiplier: Dominant brands in the top quintile of popularity are cited 1.4x to 2.8x more frequently than the average brand.
- Mention Rates: As shown in Figure 2, the probability of a brand being mentioned climbs steeply with its real-world popularity, regardless of whether the engine is allowed to search the web.
- Ranking Penalty: In Study 2, using an invented brand name instead of the original real brand caused a product to drop an average of 0.44 positions on a 1-5 scale.
- The Persona Effect: Interestingly, giving the AI a specific "buyer persona" (e.g., a budget-conscious parent) was the only intervention that successfully reduced the popularity bias, as it forced the engine to look for specific attributes rather than just "the best-known" options.
Limitations and Future Work
The authors acknowledge several constraints:
- Black-Box APIs: Because they used commercial APIs, they couldn't see the hidden system prompts or internal reranking logic used by the providers.
- Geographic Focus: The study focused on Italian-language queries and the Italian market; results might vary across different cultures or languages.
- Static Snapshot: LLMs and search indexes are updated constantly; this study represents a snapshot in time (late 2024).
Future Work includes testing "cold-start" scenarios with real-world brands founded after the models' training cutoff dates and performing sentiment analysis to see if popular brands are not just mentioned more, but also described more favorably.
Summary updated