TL;DR
This paper introduces a novel unsupervised approach to abstractive multi-document summarization that jointly performs sentence fusion and paraphrasing. By leveraging word embeddings and a graph-based sentence representation, their method generates more abstractive and informative summaries than existing techniques, while also addressing the challenge of summary length constraints.
What Problem Does This Paper Address?
Creating concise and informative summaries from multiple documents is a challenging task. Traditional extractive summarization methods, which select existing sentences, often lead to redundant or biased outputs. Abstractive summarization, which generates new sentences, is more desirable but significantly harder, especially in a multi-document setting where information overlap is common. Existing abstractive methods often rely on complex neural networks that require large amounts of training data, or they focus on sentence compression (removing words) rather than true paraphrasing (rewriting with new words). This paper tackles the problem of unsupervised abstractive multi-document summarization, aiming to achieve high-quality summaries without relying on labeled training data, and to go beyond simple sentence compression by incorporating paraphrasing.
Methods
The core of this work is a Paraphrastic Sentence Fusion Model. This model operates in two main stages:
- Word Graph Construction for Sentence Fusion: For a cluster of related sentences, a word graph is built. Nodes represent words (with POS tags), and directed edges connect adjacent words. Sentences are integrated by mapping words to existing nodes if they match in form and POS tag. This graph allows for the generation of multiple "fused" sentence candidates by finding shortest paths from a start to an end node.
- Candidate Ranking and Paraphrasing:
- Sentence Embedding: Each candidate sentence is embedded into a vector space using a bi-directional GRU (Gated Recurrent Unit) network, leveraging pre-trained word2vec embeddings.
- Candidate Ranking: An undirected graph is constructed where candidate sentences are vertices. Edges are weighted based on the cosine similarity of their sentence embeddings. The TextRank algorithm is then applied to this graph to rank the candidates by importance.
- Context-Sensitive Lexical Substitution: To achieve abstractiveness beyond simple deletion, the model identifies nouns and verbs in the word graph for potential substitution. It uses the PPDB 2.0 dataset to find paraphrases and a context-sensitive scoring mechanism (addCos) to select the best substitutes.
- Confidence Score: A 3-gram language model is used to assign a confidence score to the generated sentences, ensuring grammaticality.
- Final Scoring: A combined score is calculated using the candidate's rank score and its abstractiveness score (derived from addCos and the confidence score), allowing for the selection of N-best paraphrastic fusions.
For the Multi-Document Abstractive Summarization system, the authors employ:
- Sentence Clustering: Hierarchical agglomerative clustering with a complete linkage criterion is used to group similar sentences based on their embeddings. This helps reduce redundancy and improve information coverage.
- Abstractive Sentence Selection: An Integer Linear Programming (ILP) framework is used to select the best subset of generated abstractive sentences. This framework aims to maximize coverage of keyphrases (extracted using RAKE) and the model's own sentence scores, while adhering to a length constraint and avoiding redundancy.
Datasets
- Sentence Level Experiments: The authors use the human-generated sentence fusion dataset released by McKeown et al. (2010), which consists of 300 English sentence pairs from newswire clusters with human-produced fusions.
- Multi-Document Level Experiments:
- DUC 2004: A standard benchmark dataset for multi-document summarization, containing 50 document clusters with 10 news articles each.
- Opinosis 1.0: A dataset of short user reviews from various sources, well-suited for multi-document summarization.
Key Insights and Ideas
- Joint Fusion and Paraphrasing: The paper's central innovation is the simultaneous handling of sentence fusion and lexical paraphrasing. This allows for more creative and abstractive summaries than methods that only compress or only fuse.
- Graph-based Sentence Representation: Using a word graph to represent related sentences and then finding shortest paths is an effective way to generate diverse fusion candidates.
- Leveraging Embeddings for Similarity: Employing sentence embeddings derived from bi-directional GRUs for candidate ranking is crucial for capturing semantic similarity, overcoming the limitations of purely lexical overlap-based methods.
- Addressing Length Constraints: The ILP formulation for sentence selection explicitly incorporates a length limit, offering an optimal solution to a long-standing problem in summarization.
- Unsupervised Approach: The entire system is unsupervised, making it applicable to domains where large labeled datasets are scarce.
Key Results
- Sentence Level: The "Paraphrastic Fusion (ours)" model achieved a higher METEOR-E score (0.43) and a significantly lower Copy Rate (76.2%) compared to baselines like Filippova (2010), Boudin and Morin (2013), and Banerjee et al. (2015). This indicates a greater degree of abstractiveness and paraphrasing.
- Multi-Document Level: The "ParaFuse_doc (ours)" model achieved the best summarization performance across various ROUGE metrics (ROUGE-1, ROUGE-2, ROUGE-WE) on both the DUC 2004 and Opinosis 1.0 datasets. The ROUGE-WE scores, which consider word embeddings, show a slight increase, further supporting the abstractive nature of the summaries.
Limitations and Future Work
- Limitations:
- The paper acknowledges that while their model aims for abstractiveness, the BLEU score is slightly lower than some baselines, which is attributed to BLEU's reliance on surface-level lexical overlap.
- The unsupervised nature, while a strength, might limit performance compared to supervised models trained on massive datasets.
- Future Work:
- The authors suggest that their sentence-level model is well-suited for full abstractive multi-document summarization.
- The ILP formulation for length constraints is presented as an optimal solution, implying potential for further refinement or application in other summarization tasks.
Summary updated