Modeling Endorsement for Multi-Document Abstractive Summarization

Logan Lebanoff, Bingqing Wang, Zhe Feng, Fei Liu

Abstract

A crucial difference between single- and multi-document summarization is how salient content manifests itself in the document(s). While such content may appear at the beginning of a single document, essential information is frequently reiterated in a set of documents related to a particular topic, resulting in an endorsement effect that increases information salience. In this paper, we model the cross-document endorsement effect and its utilization in multiple document summarization. Our method generates a synopsis from each document, which serves as an endorser to identify salient content from other documents. Strongly endorsed text segments are used to enrich a neural encoder-decoder model to consolidate them into an abstractive summary. The method has a great potential to learn from fewer examples to identify salient content, which alleviates the need for costly retraining when the set of documents is dynamically adjusted. Through extensive experiments on benchmark multi-document summarization datasets, we demonstrate the effectiveness of our proposed method over strong published baselines. Finally, we shed light on future research directions and discuss broader challenges of this task using a case study.

Introduction

Summarizing a cluster of documents—such as multiple news articles covering the same event—presents unique challenges compared to single-document summarization. Modern models often struggle with memory constraints when processing long concatenations of articles and frequently fall victim to "lead bias," where they over-prioritize the beginning of documents. In multi-document contexts, salience is better defined by the endorsement effect: the idea that information repeated across different sources is the most important to include in a final summary.

Methodology

We propose a two-stage framework that explicitly models this endorsement effect to identify and prioritize salient information.

Stage 1: Modeling the Endorsement Effect

First, the system generates a brief synopsis for every document in the cluster using a pre-trained summarizer (e.g., BART). These synopses act as "endorsers." We then align these synopses with the original documents to identify which segments of the source text are supported by other documents.

The endorsement process: Synopses from multiple documents (Doc B, Doc C) serve as endorsers to identify salient segments in a target document (Doc A).

To ensure the model identifies meaningful units rather than isolated words, we utilize a maximum sum subarray algorithm to extract contiguous segments of endorsed text. This fine-grained approach allows the model to distinguish between essential facts and surrounding "filler" clauses.

Stage 2: Summarization with Companion Heads

We introduce a Companion Head architecture within a Transformer-based encoder-decoder model. While standard attention heads look at the entire source text, specialized companion heads are restricted to tokens that have received a high level of endorsement (e.g., mentioned by at least two other synopses). By interpolating these heads, the model is forced to focus on information that the group of documents agrees is important.

Experimental Results

We evaluated our model on the WCEP (Wikipedia Current Events Portal) dataset for in-domain performance, and on DUC-04 and TAC-11 to test out-of-domain generalization.

Performance on WCEP

Our "Endorser-Sequential" model, which prioritizes endorsements from later documents to filter out redundant background information, significantly outperformed both extractive and regression-based baselines.

Performance comparison on the WCEP dataset. Our Endorser-Sequential method achieves state-of-the-art ROUGE scores compared to strong extractive baselines.

As shown in the results, the Endorser-Sequential model achieved a ROUGE-1 score of 45.4, a substantial improvement over the strongest previous baseline (TSR at 35.3). Furthermore, the model demonstrated strong generalization capabilities, performing competitively on DUC-04 and TAC-11 without specific fine-tuning on those domains.

Discussion and Future Work

A key advantage of this method is its dynamic adjustability. Because endorsement is handled at the attention level, documents can be added or removed from a cluster on the fly without retraining the network.

However, the system's performance is currently tied to the quality of the initial synopses. Future research could explore weighting endorsements based on source trustworthiness or automatically learning the optimal parameters for the companion heads. This work lays the foundation for more robust, salience-aware multi-document summarization systems that better mimic human information synthesis.

Summary updated

Abstract

A crucial difference between single- and multi-document summarization is how salient content manifests itself in the document(s). While such content may appear at the beginning of a single document, essential information is frequently reiterated in a set of documents related to a particular topic, resulting in an endorsement effect that increases information salience. In this paper, we model the cross-document endorsement effect and its utilization in multiple document summarization. Our method generates a synopsis from each document, which serves as an endorser to identify salient content from other documents. Strongly endorsed text segments are used to enrich a neural encoder-decoder model to consolidate them into an abstractive summary. The method has a great potential to learn from fewer examples to identify salient content, which alleviates the need for costly retraining when the set of documents is dynamically adjusted. Through extensive experiments on benchmark multi-document summarization datasets, we demonstrate the effectiveness of our proposed method over strong published baselines. Finally, we shed light on future research directions and discuss broader challenges of this task using a case study.
BibTeX
@misc{lebanoff2021modeling,
  author = {Logan Lebanoff and Bingqing Wang and Zhe Feng and Fei Liu},
  title = {{Modeling Endorsement for Multi-Document Abstractive Summarization}},
  year = {2021},
  eprint = {2110.07844},
  archivePrefix = {arXiv},
  url = {https://arxiv.org/abs/2110.07844},
}
Figure