Bias in Conversational Search: The Double-Edged Sword of the Personalized Knowledge Graph

Emma J. Gerritse, Faegheh Hasibi, Arjen P. de VriesarXiv 2020 · DOI · arXiv

TL;DR

Conversational search systems risk creating severe echo chambers because they deliver concise, single-answer responses tailored by Personalized Knowledge Graphs (PKGs). This position paper outlines how PKGs act as a double-edged sword—both amplifying and potentially mitigating bias—and proposes design frameworks to balance user personalization with information diversification.

What Problem Does This Paper Address?

Traditional search engines present a list of results (the classic "ten blue links"), allowing users to scan and select from diverse perspectives. Conversational AI assistants (like Alexa or Google Home), however, typically provide a single, concise answer.

When these systems use Personalized Knowledge Graphs (PKGs)—structured repositories of a user's personal data, history, and preferences—to tailor responses, they risk reinforcing existing biases. Because users naturally gravitate toward information that aligns with their prior views, optimizing purely for user satisfaction can trap them in extreme filter bubbles. The authors address the challenge of identifying, measuring, and mitigating these multi-layered biases without alienating the user.

Methods

As a position paper, this work does not introduce a new algorithm; instead, it categorizes conversational bias and proposes architectural and UI/UX strategies to handle it:

  • Bias Categorization: The authors map conversational bias into three buckets: algorithmic (e.g., data source skews during KG construction, or gender bias intensified by neural ranking models like BERT), human (demographic skews in the user base), and hybrid (the vicious feedback loop between a biased user and a biased algorithm).
  • Mitigation Frameworks: They propose three design paths:
  • Ignorant: The system ignores bias (argued to be dangerous, citing examples like Microsoft's Tay chatbot).
  • User in Control: The system detects bias and offers UI elements (like a slider) allowing users to choose their preferred degree of result diversification.
  • Reflective: A mutual feedback loop where the system explicitly flags controversial topics or misinformation and provides alternatives, while allowing users to inspect and edit their own PKG to correct system-induced biases.

The paper illustrates these strategies through conceptual interaction flows, showing how a system might handle biased phrasing or fake news:

Figure 1: Example of the User in Control strategy. The system detects the possible bias in the word ’godfathers’, and offers the user also some interesting female answers.
Figure 2: Example of the Reflective strategy, where the sys- tem detects that the user is biased towards fake news, and the system supplies them with an alternative.

Datasets

Not discussed in the paper. The authors note that existing conversational benchmarks (such as TREC CAsT and QuAC) are currently restricted to standard question-answering and are insufficient for evaluating multi-turn bias or user satisfaction in personalized settings.

Key Insights and Ideas

  • The Double-Edged Sword: While PKGs accelerate the filter-bubble effect by feeding a system highly personalized data, their structured nature is also the key to the solution. Because PKGs map explicit relationships between entities, developers can use them to detect a biased user profile and deliberately traverse alternative nodes to inject diversity.
  • Time-Based Clutter: PKG construction is highly vulnerable to temporal spikes. Major events (like elections or a pandemic) cause temporary surges in specific social media behaviors. If a system doesn't actively "clean up" this time-based clutter, a user's profile can become permanently distorted by a temporary interest.
  • The Preference vs. Bias Dilemma: Drawing the line between a benign user preference (e.g., always wanting to buy the same brand of guitar strings) and a harmful bias (e.g., believing medical misinformation or political propaganda) remains an open ethical and technical challenge.

Key Results

As this is a conceptual position paper, there are no quantitative performance metrics or experimental datasets presented. The primary qualitative contribution is the conceptual framework defining the interaction strategies (User in Control vs. Reflective) and the recommendation to use Wizard of Oz experiments (where a human secretly simulates the AI) to evaluate user tolerance for being corrected or diversified before writing code.

Limitations and Future Work

  • The Risk of User Annoyance: Aggressive debiasing or lecturing users on fake news can severely degrade user satisfaction. Users generally want a convenient search tool, not an educator, meaning over-correction could cause users to abandon the system entirely.
  • Evaluation Sparsity: The authors propose a bias-measurement method that involves swapping nodes to an "opposite polarity" (e.g., swapping left-wing and right-wing political nodes) to see how much the output changes. However, they acknowledge that "opposite" information does not exist for all nodes in a standard knowledge graph, limiting how well this technique generalizes.
  • Future Directions: The authors call on the community to design explicit user studies to find the exact threshold where diversification compromises user engagement, and to build data-driven models capable of dynamically implementing these UI controls.

Summary updated

Abstract

Conversational AI systems are being used in personal devices, providing users with highly personalized content. Personalized knowledge graphs (PKGs) are one of the recently proposed methods to store users' information in a structured form and tailor answers to their liking. Personalization, however, is prone to amplifying bias and contributing to the echo-chamber phenomenon. In this paper, we discuss different types of biases in conversational search systems, with the emphasis on the biases that are related to PKGs. We review existing definitions of bias in the literature: people bias, algorithm bias, and a combination of the two, and further propose different strategies for tackling these biases for conversational search systems. Finally, we discuss methods for measuring bias and evaluating user satisfaction.
BibTeX
@misc{gerritse2020bias,
  author = {Emma J. Gerritse and Faegheh Hasibi and Arjen P. de Vries},
  title = {{Bias in Conversational Search: The Double-Edged Sword of the Personalized Knowledge Graph}},
  year = {2020},
  eprint = {2010.10409},
  archivePrefix = {arXiv},
  url = {https://arxiv.org/abs/2010.10409},
}
Figure