Introduction
Extractive summarization aims to identify and select the most salient sentences from a document to create a concise summary. Traditional state-of-the-art models typically process documents in a single pass, reading from start to finish before making a final selection. However, this "one-and-done" approach contrasts sharply with human cognitive processes. When summarizing complex texts, humans often browse for a general gist, then re-read specific sections to refine their understanding. This paper introduces Iterative Text Summarization (ITS), a framework designed to mimic this reflective behavior by polishing document representations through multiple reading passes.
The ITS Framework
The core of the ITS model is its ability to iteratively refine its internal understanding of a document. The architecture consists of three primary components that operate in a loop:
- Selective Reading Module: Unlike standard Gated Recurrent Units (GRUs), this module incorporates a specialized gating mechanism. It takes the current sentence and the global document representation from the previous iteration as inputs, deciding which information to update or retain based on its relevance to the evolving "gist."
- Iterative Unit: This component updates the global document representation ($D_k$) after each pass, effectively "refreshing" the model's context.
- Sentence Labeling Module: To make final predictions, the model concatenates hidden states from all iterations. A multi-layer perceptron (MLP) analyzes this "history of understanding" to assign importance scores.
Experimental Results
The ITS model was evaluated on two benchmark datasets: CNN/DailyMail for large-scale supervised learning and DUC2002 for testing cross-domain generalization.
Quantitatively, ITS consistently outperformed previous state-of-the-art extractive models such as SummaRuNNer and Hybrid MemNet. On the CNN/DailyMail dataset, it achieved a Rouge-1 score of 27.4 (at 75 bytes), representing a significant improvement over the 26.2–26.3 range of its predecessors. Furthermore, the model maintained its performance lead on DUC2002, suggesting that the iterative mechanism learns robust reading strategies rather than just memorizing dataset-specific patterns.
Analysis: The Power of Polishing
A key insight of this work is the "polishing" effect observed over multiple iterations. The authors found that performance improves steadily up to approximately five iterations before beginning to decline, suggesting a "sweet spot" where the model has sufficiently filtered noise without over-analyzing.
As shown in the heatmaps, early iterations often assign similar probabilities to many sentences. As the process continues, the selective reading module acts as a contextual filter. By comparing sentences to the previous iteration's summary, the model identifies redundant information and focuses on truly salient content, leading to a "saturation" of focus on the best summary candidates.
Conclusion
The ITS framework demonstrates that "reading twice" significantly enhances an AI's ability to extract meaningful summaries. While the iterative approach introduces some computational overhead, the gains in summary quality and generalization are substantial. Future work may explore applying this iterative logic to abstractive summarization or implementing dynamic "early exit" strategies to optimize computational efficiency.
Summary updated