Exploring the Escalation of Source Bias in User, Data, and Recommender System Feedback Loop

Yuqi Zhou, Sunhao Dai, Liang Pang, Gang Wang, Zhenhua Dong, Jun Xu, Ji-Rong Wen

TL;DR

Recommender systems exhibit a significant "source bias," consistently ranking AI-generated content (AIGC) higher than human-written equivalents. This creates a dangerous feedback loop: as users interact with more AIGC, the models retrain on this biased data, further amplifying the preference and eventually leading to a decline in overall recommendation quality. The authors propose a novel debiasing method using L1-loss optimization to align the embedding spaces of human and AI content, restoring balance to the digital ecosystem.

What Problem Does This Paper Address?

As Large Language Models (LLMs) flood the internet with high-quality, low-cost text, recommender systems (RS) have become the primary gatekeepers for this content. However, recent studies suggest these systems aren't neutral; they often suffer from source bias, where the underlying models (like BERT or RoBERTa) favor the stylistic patterns of LLM-generated text over human writing, even when the semantic meaning is identical.

The authors identify a critical "feedback loop" problem. If a system ranks AIGC higher, users are more likely to click it. This interaction data then becomes training data for the next version of the model, which learns to favor AIGC even more strongly. This isn't just a fairness issue for human creators; the paper warns that an ecosystem dominated by AIGC can eventually lead to "model collapse," where the recommendation performance itself degrades due to the lack of diverse, human-originated data.

Figure 1: Model preference grows over time with the feedback loop of humans, data, and the recommender system. The red color is used for HGC icon and the blue color is used for AIGC icon. The subsequent figures use the same color scheme.

Methods

The researchers explored the problem across three phases: HGC Dominate (the past), HGC-AIGC Coexist (the present), and AIGC Dominate (the future).

To study the long-term effects, they built a realistic simulator using various click models (like the Position-Based Model) to mimic how users interact with ranked lists. They then developed a black-box debiasing method that doesn't require the system to explicitly "detect" AI content. Instead, their approach involves:

  1. Uniform Rewriting: Prompting an LLM to rewrite all training data into AIGC copies.
  2. Embedding Alignment: Using an L1-loss function to minimize the distance between the embeddings of original human-generated content (HGC) and its AIGC counterpart.
  3. Entropy Constraint: Adding a constraint to ensure that while HGC and AIGC versions of the same item are close together, different items remain distinct in the embedding space to preserve ranking performance.

Datasets

The study utilized the Amazon product dataset, focusing on three diverse categories: Health, Beauty, and Sports.

  • AIGC Generation: They used mainstream LLMs—ChatGPT (gpt-3.5-turbo), Llama 2, Mistral, and Gemini Pro—to rewrite product descriptions while maintaining semantic equivalence.
  • Verification: A human evaluation study confirmed that users generally found the HGC and AIGC descriptions to be of equal quality, ensuring that any ranking preference found was due to model bias rather than a genuine difference in content utility.
Figure 3: Quality verification of the constructed datasets through human evaluation.

Key Insights and Ideas

  • The "AIGC Premium": Even when an LLM-generated description is semantically identical to a human one, popular sequential models (like SASRec and BERT4Rec) consistently rank the AI version higher.
  • Bias Escalation: The bias isn't static. In the "AIGC Dominate" phase, the preference for AI content is pushed to the top of the ranking list through the iterative feedback loop.
  • Performance Degradation: There is a "tipping point." While a little AIGC might not hurt, a content ecosystem dominated by it (e.g., after 20 iterations of the feedback loop) results in a measurable drop in recommendation accuracy (NDCG).
  • Alignment over Detection: The most effective way to fix the bias isn't to ban AI content, but to force the model's internal representations (embeddings) of HGC and AIGC to be indistinguishable.

Key Results

  • Bias Measurement: The authors used "Relative $\Delta$" to measure bias. In the Health dataset, models like GRU4Rec showed a strong initial preference for AIGC (Relative $\Delta \approx -23\%$).
  • Feedback Loop Impact: Without intervention, the bias amplified significantly over 10 iterations. In some scenarios, the absolute Relative $\Delta$ increased by over 100 points.
  • Debiasing Success: The proposed L1-loss method successfully reduced the Relative $\Delta$ to near zero (or even slightly positive, favoring HGC) across all datasets and models.
  • Performance Boost: Interestingly, the debiasing method often improved standard recommendation metrics (like NDCG@3). By forcing the model to ignore "source noise," it became better at identifying true semantic relevance.

Limitations and Future Work

The authors acknowledge that their study primarily focuses on textual AIGC. As multimodal content (AI-generated images and videos) becomes more common, the source bias in those domains—and how they interact with text—remains an open question. Additionally, the study assumes that the LLM-rewritten content is semantically equivalent to the human original; however, in the real world, LLM "hallucinations" could introduce factual inaccuracies that this specific debiasing method might not address. Future work could explore more complex user behavior models and the impact of AIGC on non-sequential recommendation architectures.

Summary updated

Abstract

Recommender systems are essential for information access, allowing users to present their content for recommendation. With the rise of large language models (LLMs), AI-generated content (AIGC), primarily in the form of text, has become a central part of the content ecosystem. As AIGC becomes increasingly prevalent, it is important to understand how it affects the performance and dynamics of recommender systems. To this end, we construct an environment that incorporates AIGC to explore its short-term impact. The results from popular sequential recommendation models reveal that AIGC are ranked higher in the recommender system, reflecting the phenomenon of source bias. To further explore the long-term impact of AIGC, we introduce a feedback loop with realistic simulators. The results show that the model's preference for AIGC increases as the user clicks on AIGC rises and the model trains on simulated click data. This leads to two issues: In the short term, bias toward AIGC encourages LLM-based content creation, increasing AIGC content, and causing unfair traffic distribution. From a long-term perspective, our experiments also show that when AIGC dominates the content ecosystem after a feedback loop, it can lead to a decline in recommendation performance. To address these issues, we propose a debiasing method based on L1-loss optimization to maintain long-term content ecosystem balance. In a real-world environment with AIGC generated by mainstream LLMs, our method ensures a balance between AIGC and human-generated content in the ecosystem. The code and dataset are available at https://github.com/Yuqi-Zhou/Rec_SourceBias.
BibTeX
@inproceedings{zhou2025exploring,
  author = {Yuqi Zhou and Sunhao Dai and Liang Pang and Gang Wang and Zhenhua Dong and Jun Xu and Ji-Rong Wen},
  title = {{Exploring the Escalation of Source Bias in User, Data, and Recommender System Feedback Loop}},
  booktitle = {International ACM SIGIR Conference on Research and Development in Information Retrieval},
  year = {2025},
  doi = {10.1145/3726302.3729972},
}
Figure