In text summarization, researchers have long navigated a tension between two distinct philosophies: extraction and abstraction. Extractive models act like highlighters, selecting verbatim sentences from a source to ensure factual reliability. Abstractive models act like writers, generating novel phrasing to create fluid, concise summaries.
While abstractive models—powered by modern Transformers—produce more human-like prose, they suffer from "hallucinations," generating content that is either logically inconsistent with the source or entirely fabricated. This has led to a central research challenge: how to increase a model's creative flexibility without sacrificing its factual grounding.
The Pareto Frontier of Faithfulness
The core difficulty in the field is that faithfulness and abstractiveness often move in opposite directions. As Faisal Ladhak and Esin Durmus observe in Faithful or Extractive?, many models "cheat" to achieve high faithfulness scores by simply becoming more extractive. When a model is forced to be more abstractive, the risk of factual distortion increases because the model must synthesize information rather than just relocate it.
This trade-off is not just a matter of linguistic style but of model objective. Meng Cao and Yue Dong note in Learning with Rejection for Abstractive Text Summarization that constraining models to be faithful often causes them to revert to copying, losing the benefits of synthesis and conciseness.
Why Models Hallucinate
Hallucinations are often categorized into two types:
- Intrinsic: The model misinterprets the source (e.g., swapping a subject and object).
- Extrinsic: The model introduces outside information.
Interestingly, John Wieting and Yue Dong argue in Faithful to the Document or to the World? that extrinsic hallucinations aren't always "errors." Often, models use pre-trained world knowledge to fill in gaps, creating summaries that are factually true in the real world but "unfaithful" to the specific document provided. Conversely, Zhichao Xu shows that models often favor these internal biases over the provided text, a problem that can be mitigated through Context-aware Decoding (CAD), which mathematically penalizes the model for ignoring the context Context-aware Decoding Reduces Hallucination in Query-focused Summarization.
Mechanisms for Grounding
To break the trade-off, researchers have introduced several "grounding" mechanisms that force abstractive models to respect the source's logic:
- Natural Language Inference (NLI): Tobias Falke and Ido Dagan explored using NLI to rerank candidate summaries, ensuring the source document logically entails the summary Ranking Generated Summaries by Correctness. Similarly, Jiajun Zhang incorporated entailment knowledge directly into the training process Ensure the Correctness of the Summary.
- Knowledge Graphs: Chenguang Zhu proposed FASUM, which uses relational triples (subject-relation-object) to anchor the generation process in hard facts Enhancing Factual Consistency of Abstractive Summarization.
- Inverse Reinforcement Learning (IRL): Deyi Xiong and Yue Dong used IRL to let models "learn" the optimal balance between conciseness and accuracy by imitating human experts, rather than relying on static loss functions Inverse Reinforcement Learning for Text Summarization.
The Hybrid Middle Ground
A significant portion of the literature seeks a "third way": models that extract first and then refine. Jiacheng Xu introduced JECS, which extracts sentences and then uses syntactic compression to prune them, avoiding the "hallucination" risk of generating from scratch Neural Extractive Text Summarization with Syntactic Compression.
Similarly, David Konopnicki and Haggai Roitman proposed EditNet, which mimics a human editor by deciding whether to keep, rephrase, or reject candidate sentences An Editorial Network for Enhanced Document Summarization. These hybrid approaches acknowledge that while pure abstraction is the goal, the factual safety of extraction remains a necessary foundation.
Rethinking Evaluation
Finally, the research suggests that our metrics may be part of the problem. Traditional metrics like ROUGE prioritize word overlap, which can penalize highly faithful but abstractive summaries. Mohit Bansal and David Wan point out that even extractive models can be "unfaithful" if they strip away context, leading to dangling pronouns or broken logic Extractive is not Faithful. This has spurred the development of reference-free metrics like BooookScore Yapei Chang and EXTEVAL, which focus on coherence and logical grounding rather than simple lexical similarity.