Spiral of Silence: How is Large Language Model Killing Information Retrieval?
- A Case Study on Open Domain Question Answering
Abstract
As Large Language Models (LLMs) increasingly populate the internet with synthetic text, Retrieval-Augmented Generation (RAG) systems face a critical feedback loop. This study explores the "Spiral of Silence" effect, where search engines and retrieval algorithms prioritize AI-generated content over human-authored text. Through iterative simulations, we demonstrate that this bias leads to a collapse in information diversity and a long-term decline in retrieval effectiveness, potentially marginalizing human voices in the digital ecosystem.
Introduction: The Post-Human Web
With projections suggesting that up to 90% of online content could be synthetic by 2026, the integrity of the "knowledge base" used by RAG systems is under threat. These systems rely on searching the web to provide grounded answers. If the web becomes a "hall of mirrors" filled with LLM-generated responses, the quality of retrieved information may degrade. This paper investigates the "Spiral of Silence": a phenomenon where human-contributed information is algorithmically marginalized because retrievers favor the structured, highly relevant style of AI-generated text.
Methodology: Simulating the Information Ecosystem
To model the evolution of the digital landscape, the researchers developed an iterative simulation pipeline.
The process involves four key stages:
- Baseline: Establishing a standard dataset of human-authored documents ($D_0$).
- Generation: Using an LLM to answer questions based on the current dataset.
- Injection: Adding these LLM-generated "answer documents" back into the search index.
- Iteration: Repeating this cycle for 10 iterations to simulate long-term web growth.
The study utilized four gold-standard Open Domain Question Answering (ODQA) benchmarks: NQ (Natural Questions), WebQ (WebQuestions), TriviaQA, and PopQA (focused on long-tail knowledge). Performance was evaluated across multiple retrieval methods, including BM25, Contriever, and BGE.
Key Insights: The AI Bias in Search
The simulations revealed several critical trends:
- Algorithmic Bias: Both traditional keyword-based (BM25) and modern neural retrievers consistently rank LLM-generated text higher than human text. The "cleaner" and more direct nature of AI text is often misinterpreted as higher quality by ranking algorithms.
- The Vanishing Human: Human-authored content in the top 50 search results plummeted from 100% to less than 10% after ten iterations. This demonstrates the "Spiral of Silence" in action—human voices are algorithmically evicted from top results.
- Information Homogenization: Diversity of information, measured by Self-BLEU scores, collapsed over time. Search results became an echo chamber of repetitive AI "opinions," even when factually incorrect.
Results and Performance Decay
The impact on retrieval performance is complex. While adding LLM text can provide a short-term "sugar high" boost in retrieval scores (e.g., a 31.2% jump in Acc@5 for BM25 on TriviaQA), the long-term trend is one of systemic decay.
As shown in the results, retrieval effectiveness generally trends downward across most models and datasets over multiple iterations. Furthermore, when misinformation is introduced, the "Spiral of Silence" accelerates, as RAG systems quickly prioritize and propagate incorrect AI-generated "facts."
Conclusion and Future Directions
While the simulation is a simplified model of the web, it highlights a significant risk for the future of information retrieval. Mitigation strategies like Source Filtering (detecting AI text) and Content Filtering (enforcing diversity) can slow the decay but not stop it entirely. There is an urgent need for "AI-resilient" retrieval models that can preserve the authentic digital information landscape and prevent the total marginalization of human-authored content.
Summary updated