Annotating and Modeling Fine-grained Factuality in Summarization

Tanya Goyal, Greg Durrett

Abstract

Current abstractive summarization models frequently generate non-factual content, a phenomenon known as hallucination. While factuality detectors are often trained on synthetic data, this paper demonstrates a significant mismatch between synthetic errors and those produced by real models. By introducing a fine-grained human annotation schema and a Dependency Arc Entailment (DAE) model, the authors show that modeling word-to-word relationships is essential for accurate factuality detection and for training more reliable summarizers.

Introduction

Abstractive summarization aims to produce concise, fluent summaries in a model's own words. However, a major hurdle is the tendency of these models to hallucinate—generating text that is fluent but factually unsupported by the source. To address this, "critic" models are typically trained to detect errors. Due to the scarcity of human-labeled data, these critics often rely on synthetic data generated through simple heuristics like entity swapping. This paper investigates whether such synthetic data truly represents the errors made by state-of-the-art models and proposes a more granular approach to factuality.

Methodology: Dependency Arc Entailment (DAE)

The core contribution of this work is a shift from sentence-level labels to fine-grained, arc-level analysis. The researchers propose the Dependency Arc Entailment (DAE) model. Instead of a binary "factual/non-factual" judgment for a whole sentence, DAE decomposes a sentence into its grammatical dependency structure. Each arc, representing a relationship between words, is then evaluated for entailment against the source text.

Comparison of synthetic training data (Entity-centric and Generation-centric) versus fine-grained human annotation. Red arcs indicate non-factual dependencies.

To support this modeling, the authors developed:

  1. A New Error Taxonomy: Categorizing factual errors into Entity-related (wrong names/dates), Event-related (wrong actions or agents), Noun Phrase-related (wrong modifiers), and others.
  2. DAE-Weak: A weak supervision method that allows training fine-grained models using only coarse sentence-level labels by treating error locations as latent variables.
  3. Factuality-Aware Training: A technique to "mask" non-factual tokens identified by the DAE model during the training of summarization models, effectively preventing the model from learning from incorrect associations.

The Synthetic-Real Gap

A key insight of the study is the "Synthetic Mismatch." Synthetic datasets, such as those created by swapping entities (Ent-C), are dominated by simple errors. In contrast, real-world models—especially on highly abstractive datasets like XSum—produce complex "event-related" errors involving incorrect actions or agents that synthetic generators fail to mimic.

Furthermore, the study reveals that domain matters significantly; errors in CNN/Daily Mail are mostly intrinsic (mixing up information already in the text), whereas XSum errors are mostly extrinsic (bringing in outside information). Surprisingly, the authors found that up to 75% of "gold" summaries in standard benchmarks like XSum may contain non-factual information, suggesting that standard training sets are inherently noisy.

Experimental Results

The researchers evaluated their approach on the CNN/Daily Mail and XSum benchmarks with the following results:

  • Detection Performance: On the challenging XSum dataset, the fine-grained DAE model achieved 78.7% accuracy, significantly outperforming standard sentence-level factuality models (65.6%).
  • Localization: The model successfully pinpointed specific word-level errors with a 74.7% recall.
  • Summarization Quality: By using the DAE model to mask non-factual tokens during training, the researchers reduced the Word Error Rate of their summarizer from 32.2 to 23.7. Human evaluators confirmed that summaries produced this way were significantly more factual.

Conclusion

This work highlights that the granularity of labels is critical for factuality. While synthetic data provides a starting point, it often fails to capture the complexity of actual model hallucinations. Fine-grained human annotations and dependency-level modeling offer a path toward more robust factuality detectors and, ultimately, more trustworthy AI summarization systems. Future research will focus on generating more realistic synthetic data and developing methods for automatic post-hoc error correction.

Tanya Goyal, Greg Durrett. Proceedings of the 2021 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies. 2021.

Summary updated

BibTeX
@inproceedings{goyal2021annotating,
  author = {Tanya Goyal and Greg Durrett},
  title = {{Annotating and Modeling Fine-grained Factuality in Summarization}},
  booktitle = {Conference of the North American Chapter of the Association for Computational Linguistics},
  year = {2021},
  url = {https://openreview.net/forum?id=zxebmsfx6Am},
}
Figure