Redundancy reduction for multi-document summaries using A* search and discriminative training

Ahmet Aker, Trevor Cohn, Robert J. GaizauskasE-LKR 2012 · OpenReview

Abstract

Extractive multi-document summarization (MDS) often suffers from content redundancy when multiple source documents cover the same information. This paper introduces a framework that integrates a dynamic, automatically learned redundancy threshold directly into an A* search and discriminative training process. By optimizing for global evaluation metrics rather than just local sentence quality, the proposed method significantly enhances summary density and readability.

Introduction

The primary goal of extractive MDS is to select the most informative sentences from a document collection to form a concise summary. This task involves two interdependent challenges: searching for the optimal sentence combination and training the parameters that define summary quality. Historically, these steps were decoupled, often leading to summaries that, while containing relevant facts, were plagued by repetition.

While previous approaches utilized redundancy thresholds to filter similar content, these parameters were typically set manually. This research investigates whether an integrated model can learn the optimal redundancy threshold ($R$) directly from data, thereby improving the global coherence of the output.

Methodology: Learning the Redundancy Threshold

The authors extend an existing framework utilizing A search for navigation and Minimum Error Rate Training (MERT) for weight optimization. The core innovation is the introduction of a "Jump" function within the A search. As the algorithm constructs a summary, it evaluates the n-gram overlap between candidate sentences and the existing summary. If the similarity exceeds the threshold $R$, the candidate is discarded.

Flowchart of the iterative learning process for the redundancy threshold R and weights W.

Rather than hardcoding $R$, the system employs an iterative learning process. It evaluates a range of thresholds ($R, R+0.1, R-0.1$), generates candidate summaries, and uses MERT to optimize scoring weights. The threshold that yields the highest ROUGE score is selected for the next iteration, allowing the model to converge on a mathematically optimal value for specific evaluation metrics.

Experimental Setup

The model was evaluated using an image-text corpus of 296 located objects. The dataset provided web documents as inputs and up to four manually extracted model summaries (190–210 words) as ground truth. The data was split into 191 document sets for training and 105 for testing.

Results and Discussion

The proposed method, Jump with Redundancy Training (JRT), demonstrated statistically significant improvements over the baseline across both quantitative and qualitative measures.

Quantitative Performance

As shown in the table below, JRT achieved higher ROUGE-2 and ROUGE-SU4 scores, indicating better alignment with human-authored summaries.

Comparison of ROUGE-2 and ROUGE-SU4 scores between the baseline and the proposed JRT method.

Interestingly, the model learned different thresholds depending on the metric: $0.5338$ for ROUGE-2 and a stricter $0.4675$ for ROUGE-SU4. This suggests that metrics sensitive to spaced-out similarities require more aggressive redundancy filtering.

Qualitative Impact

Human evaluation revealed a dramatic increase in summary quality. The percentage of summaries receiving high scores (4 or 5 on a 5-point scale) improved significantly: clarity jumped from 47.9% to 95.9%, focus from 39.5% to 87.7%, and redundancy reduction from 12.5% to 69.4%.

The model effectively learned to distinguish between redundant facts (which should be removed) and redundant referring expressions (like proper names), which are often necessary for factual clarity.

Conclusion

By incorporating redundancy constraints directly into the search path, the JRT model produces information-dense, highly readable summaries. Future work will focus on moving beyond surface-level lexical overlap to semantic redundancy detection using embeddings, as well as improving the narrative flow to move beyond a "bag of sentences" structure.

Summary updated

BibTeX
@inproceedings{aker2012redundancy,
  author = {Ahmet Aker and Trevor Cohn and Robert J. Gaizauskas},
  title = {{Redundancy reduction for multi-document summaries using A* search and discriminative training}},
  booktitle = {E-LKR},
  year = {2012},
  url = {https://openreview.net/forum?id=zMmdQAxa0s},
}
Figure