TL;DR
This paper tackles the persistent problem of factual inconsistency (hallucination) in text summarization by leveraging natural language feedback. The authors introduce DeFacto, a high-quality dataset containing human demonstrations and detailed feedback (explanations, evidence, and instructions) on system-generated summaries. They demonstrate that fine-tuned language models can successfully use this feedback to correct factual errors, and that training a model to generate feedback before editing acts like a chain-of-thought that significantly boosts performance.
What Problem Does This Paper Address?
While modern abstractive summarization models generate highly fluent text, they frequently suffer from a lack of factual consistency—meaning they hallucinate information or misinterpret the source text.
A major root cause is the training data itself. Popular datasets like XSum are notoriously noisy; prior work shows that up to 76.9% of XSum's reference summaries contain hallucinations. Models trained on this data naturally learn to hallucinate. Gathering entirely new, massive datasets aligned with perfect factual standards is prohibitively expensive. Instead, this paper asks: Can we fix imperfect summaries by training models to follow natural language feedback and human corrections?
Methods
The authors frame summary correction as a sequence-to-sequence (Seq2Seq) problem and explore three distinct natural language generation (NLG) tasks to utilize human feedback:
- The Editing Model: Takes the input document, the initial flawed summary, and human feedback (instructions/explanations) to generate a factually corrected summary.
- The Critic Model: Takes the input document and initial summary to automatically predict the corrective human feedback.
- The Editor Model: Automatically corrects summaries end-to-end by predicting both the feedback and the edited summary given the document and initial summary.
The authors evaluate these tasks by fine-tuning a family of encoder-decoder models (T5-3B, T0-3B, and T0pp) using teacher forcing, and compare them against zero-shot Large Language Models (LLMs) like GPT-3.5 and GPT-4.
Datasets
The paper introduces DeFacto (Demonstrations and Feedback for FacTual Consistency), a dataset built on top of the XSum validation and test sets. The initial summaries were generated using PEGASUS, a top-performing summarization model.
The dataset contains 2,561 data points, where human annotators performed a multi-step pipeline:
- Detected and categorized errors (intrinsic vs. extrinsic).
- Provided a natural language explanation and highlighted source text evidence.
- Wrote corrective instructions using standardized templates (e.g., Remove, Replace, Modify).
- Created a final, human-edited, factually consistent summary.
Key Insights and Ideas
- Feedback as a "Chain of Thought": The Editor model (which generates both feedback and the edit) performed better than a baseline that only attempted to output the corrected summary. Forcing the model to explain why something is wrong and how to fix it acts as an intermediate reasoning step that improves final generation quality.
- Error Type Dictates the Fix: Fine-grained analysis of DeFacto revealed a clear mapping between error types and human edits. Extrinsic errors (hallucinating completely new info) are predominantly fixed by removing information. Intrinsic errors (misinterpreting source info) are more diverse but are most frequently fixed by replacing information.
- The LLM Controllability Gap: While zero-shot LLMs (like GPT-3.5) are highly capable of improving factual consistency, they struggle significantly with controllable text generation. They often rewrite the entire summary from scratch rather than making minimal edits, or completely miss errors—GPT-3.5 claimed "No editing instructions needed" 23.9% of the time when given a known factually inconsistent summary.
Key Results
- Factual Consistency Boost: Human-edited summaries in DeFacto achieved dramatically higher factuality scores compared to both the original system outputs and the original XSum reference summaries (measured via automatic metrics DAE and QAFactEval).
- Fine-Tuned Model Success: The fine-tuned T0pp model effectively ingested human instructions and explanations. The
D+S+I+Evariant (Document + Summary + Instruction + Explanation) achieved a DAE factuality score of 0.911 (up from the initial system's 0.704) and closely matched human edits (89.22 ROUGE-1). - Competitive End-to-End Correction: When evaluated on the entire test set (including summaries without errors), the end-to-end Editor model achieved competitive or superior performance compared to five state-of-the-art factuality-aware summarization systems (like CLIFF and FactPegasus), despite being trained on significantly less data.
Limitations and Future Work
The authors openly note several limitations:
- Scale: DeFacto is relatively small (2,561 examples) due to the complex, expensive, and time-consuming nature of human annotation.
- Diversity: The dataset is restricted to one source dataset (XSum) and one base model (PEGASUS).
- Subjectivity: Factual consistency is a relatively objective metric. Future work is required to see if natural language feedback can generalize to highly subjective user preferences, such as style, tone, or specific formatting constraints.
Summary updated