Abstract
Abstractive text summarization models often struggle with factual inconsistency, or "hallucinations," where generated summaries deviate from the source material. This paper proposes a novel approach to improve faithfulness by utilizing Elementary Discourse Units (EDUs) as a granular guidance signal. By integrating EDUs into the GSum framework using a BART-based model, the researchers demonstrate significant improvements in both summary quality (ROUGE scores) and factual accuracy (Entailment and QAGS) over existing state-of-the-art methods on the CNN/Daily Mail dataset.
Introduction
While abstractive summarization offers the potential for fluent and concise document summaries, its practical utility is limited by the tendency of models to misinterpret relationships between ideas or invent facts. Previous attempts to mitigate this issue have used "highlighted sentences" as guidance. However, sentences are often too long and contain irrelevant "noise" that can confuse the model.
This research addresses the granularity problem by proposing the use of Elementary Discourse Units (EDUs)—the smallest semantically precise units of meaning, such as clauses or phrases. The authors hypothesize that EDUs provide a cleaner, more focused signal for the model to follow, leading to more faithful summaries.
Methodology
The proposed pipeline combines three core components:
- BART: A high-performing pre-trained language model serves as the generative engine.
- EDUs (via DISCOBERT): The source text is segmented into EDUs using DISCOBERT. These units represent complete, independent thoughts, allowing for better identification of inter-sentence relations like causality.
- GSum Framework: This guided summarization framework allows the BART model to accept an additional input signal (the EDUs) to steer the generation process.
During training, the model uses "Oracle" guidance derived from human-written reference summaries. At test time, the system employs MatchSum to identify important sentences and DISCOBERT to extract the corresponding crucial EDUs, which are then fed into the BART model.
Experimental Results
The researchers evaluated their approach against several strong baselines, including CLIFF (contrastive learning) and SEASON (salience-based guidance), using the CNN/Daily Mail benchmark.
Summary Quality
The BART+EDUs(G) model demonstrated superior performance in standard quality metrics. As shown in Table 1, it achieved a ROUGE-1 score of 0.54, significantly outperforming standard BART (0.44) and the SEASON baseline (0.46).
Faithfulness and Semantic Similarity
To measure factual consistency, the authors utilized Entailment scores and QAGS (a question-answering consistency metric). The hybrid BART+Sents,EDUs(G) model, which combines sentence-level context with EDU-level precision, yielded the highest faithfulness scores in the study.
As detailed in Table 2, the hybrid model achieved a 0.929 Entailment score and a 0.093 QAGS score. Furthermore, it showed superior semantic similarity to reference summaries with a SentSim score of 0.854. These results suggest that while EDUs provide the necessary precision for faithfulness, sentences may still provide valuable contextual information.
Discussion and Conclusion
The findings highlight that granularity is key to guiding abstractive models. By breaking text into EDUs, models can better preserve the semantic integrity of the source document.
However, some limitations remain. The computational overhead of dual-encoding and EDU extraction was not extensively discussed, and the study was limited to news data. Future work should explore the scalability of EDU-based guidance in more complex domains, such as legal or medical documentation, where factual precision is paramount. Overall, this approach offers a promising path toward more reliable and trustworthy AI-generated summaries.
Summary updated