Follow the Knowledge: Structural Biases and Artefacts in Knowledge Grounded Dialog Datasets

Ehsan Lotfi, Maxime De Bruyn, Jeska Buhmann, Walter DaelemansACL 2023

TL;DR

This paper investigates how the structure and order of knowledge presented to crowd-workers in dialogue datasets might be influencing their "knowledge selection" choices, leading to potential biases. The authors demonstrate that simple, content-agnostic heuristics can achieve surprisingly competitive results, questioning the true "flow" of knowledge in these datasets and suggesting a need for better dataset curation practices.

What Problem Does This Paper Address?

Knowledge Grounded Dialog (KGD) systems aim to generate responses that are informed by external knowledge. To build these systems, researchers rely on large datasets curated through crowd-sourcing. A key task in KGD is "knowledge selection" – choosing the right piece of information to ground a response. While researchers have focused on modeling complex "knowledge flows" to improve this selection, this paper questions whether these flows are genuine conversational patterns or simply artifacts of how the knowledge is presented to crowd-workers. Specifically, they hypothesize that workers might be "following the knowledge" rather than truly understanding and integrating it.

Methods

The authors explore this hypothesis by:

  • Analyzing existing KGD datasets: They focus on two popular datasets, Wizard of Wikipedia (WoW) and Topical-Chat.
  • Proposing content-agnostic heuristics: They develop simple models that select knowledge based purely on its structure and order within the provided pool. These include:
  • Topic-First (T0): Always picks the first sentence of the dialog-topic article.
  • Topic-Next (T+): Starts with the first sentence of the dialog-topic article and proceeds to the next sentence in subsequent turns.
  • Last-Next (L+): Picks the next sentence in the passage that was selected in the previous turn.
  • Evaluating these heuristics: They measure the performance of these simple models on knowledge selection tasks.
  • Markov Model Analysis: For WoW, they further model knowledge selection as a Markov process, analyzing transition probabilities between states (e.g., choosing the dialog-topic passage vs. another, or picking the next sentence vs. another).
  • Integrating heuristics with content-aware models: They explore ways to combine these content-agnostic signals with standard transformer-based models (e.g., using them in an ensemble or as "token cues").

Datasets

  • Wizard of Wikipedia (WoW): A large-scale dataset for knowledge-grounded dialogue, where conversations are grounded on Wikipedia articles.
  • Topical-Chat: Another popular KGD dataset, featuring conversations on various entities grounded in Wikipedia articles, fun facts, and news articles.

The paper does not introduce new datasets but analyzes existing ones.

Key Insights and Ideas

  • "Following the Knowledge" is a Strong Bias: The core finding is that crowd-workers often select knowledge based on its sequential order within the provided pool, rather than complex reasoning about conversational flow. This suggests that what appears as "knowledge flow" might be an artifact of dataset curation.
  • Simple Heuristics Compete with Sophisticated Models: Surprisingly, the content-agnostic models (especially L+ for WoW) achieve performance competitive with, and sometimes exceeding, more complex, state-of-the-art models that explicitly try to model knowledge flow.
  • Dataset Curation Matters: The way knowledge options are presented (e.g., the interface, the number of options) can significantly influence annotator behavior, potentially leading to these biases.
  • Questioning Higher-Order Attributes: The results cast doubt on the significance and origin of presumed dialog-level attributes like "knowledge flow" in current crowd-sourced KGD datasets.

Key Results

  • WoW: The content-agnostic "Last-Next" (L+) model achieved competitive knowledge selection accuracy on the WoW dataset, outperforming the original baseline and rivaling some transformer-based models. Specifically, L+ achieved 28.3% accuracy on the seen test set and 30.1% on the unseen test set, compared to the baseline's 12.2% (unseen).
  • Topical-Chat: Analysis revealed a tendency to ground on Factual Sections (FS) over Article Sections (AS), with a preference for earlier parts of the article (AS1) early in conversations, and a shift towards later FS sections as conversations progress. This suggests a structural bias in grounding patterns.
  • Markov Model: The Markov model analysis for WoW showed a strong tendency to stick with the "Dialog-Topic" passage (~75% transition probability) and pick the "Next" sentence (~35% transition probability).
  • Integration: Combining content-agnostic signals with a BERT-based model (Ensemble or TokenCues) significantly improved performance over the baseline, reaching accuracies of 33.8% (seen) and 34.6% (unseen) for the ensemble model.

Limitations and Future Work

  • English-Centric: The study is limited to English datasets, overlooking datasets with more explicit annotation instructions (e.g., DuConv, NaturalConv) which might mitigate some biases.
  • Impact on Response Generation: The paper does not empirically investigate how these knowledge selection biases affect the ultimate goal of KGD: response generation. Future work could explore this using synthetic datasets.
  • Need for Better Datasets: The authors emphasize the need for better-curated datasets that consider the annotator's perspective and potentially incorporate explicit scenarios for dialog unfolding to mitigate artifacts.

Summary updated

BibTeX
@inproceedings{lotfi2023follow,
  author = {Ehsan Lotfi and Maxime De Bruyn and Jeska Buhmann and Walter Daelemans},
  title = {{Follow the Knowledge: Structural Biases and Artefacts in Knowledge Grounded Dialog Datasets}},
  booktitle = {Annual Meeting of the Association for Computational Linguistics},
  year = {2023},
}
Figure