Abstract
The most successful approaches to extractive text summarization seek to maximize bigram coverage subject to a budget constraint. In this work, we propose instead to maximize semantic volume. We embed each sentence in a semantic space and construct a summary by choosing a subset of sentences whose convex hull maximizes volume in that space. We provide a greedy algorithm based on the Gram-Schmidt process to efficiently perform volume maximization. Our method outperforms the state-of-the-art summarization approaches on benchmark datasets.
Introduction
Traditional extractive summarization—the process of selecting the most representative sentences from a document—has historically relied on discrete "bag-of-bigrams" representations. Most state-of-the-art methods focused on maximizing unique bigram coverage within a specific word budget. However, these methods often struggle with synonymy and fail to capture the broader semantic "breadth" of a document. As natural language processing has shifted toward continuous distributed representations, a need has emerged for algorithms that leverage dense semantic spaces. This paper introduces a geometric objective: Semantic Volume Maximization.
Methodology
The proposed approach moves summarization from discrete word-counting to continuous geometry.
Sentence Embedding
Sentences are first represented as vectors. The authors utilize Singular Value Decomposition (SVD) on a bigram-document matrix to project sentences into a $K$-dimensional latent space, typically using 500 or 600 dimensions.
The Geometric Objective
Rather than counting bigrams, the goal is to select a subset of sentences whose convex hull maximizes the volume in the semantic space. Intuitively, if a document's meaning is represented as a cloud of points, the summary should consist of the points that "stretch" to cover as much of that cloud's territory as possible.
The Greedy Algorithm
Finding the absolute optimal subset is computationally prohibitive. To address this, the authors employ a fast greedy algorithm based on the Gram-Schmidt process:
- Initialization: Select the sentence farthest from the document's centroid.
- Selection: Select the next sentence farthest from the first.
- Iteration: Iteratively add the sentence farthest from the subspace spanned by the already-selected sentences. This ensures each addition provides maximal "new" information.
- Termination: Stop once the word budget is reached.
Experimental Setup
The method was evaluated on two standard multi-document summarization benchmarks:
- TAC-2008: 48 problems, each containing 10 news articles.
- TAC-2009: 44 problems, each containing 10 news articles. Performance was compared against four human-written reference summaries per problem using ROUGE metrics.
Results and Discussion
The Semantic Volume approach consistently outperformed strong baselines, including Maximal Marginal Relevance (MMR) and Coverage-Based Summarization (CBS).
As shown in the results, the "Volume 600" configuration achieved a ROUGE-1 score of 37.50 on TAC-2008 and 34.37 on TAC-2009, significantly exceeding the CBS baseline. While improvements in ROUGE-2 (bigram overlap) were more modest, the superior ROUGE-1 scores suggest more complete semantic coverage.
Efficiency and Diversity
The algorithm is highly efficient, with an inference time of approximately 0.17 seconds per problem, outperforming MMR (0.38s). Geometrically, redundancy is naturally penalized: a sentence similar to one already selected will lie close to the existing subspace, adding negligible volume. This forces the algorithm to select sentences pointing in diverse semantic directions.
Conclusion
By maximizing semantic volume, this method overcomes the limitations of word-overlap models. While performance is sensitive to embedding quality and dimensionality, the geometric framework provides a robust, efficient path forward for extractive summarization using continuous representations. Future work may involve integrating more sophisticated neural sentence embeddings or adapting the model for abstractive pipelines.
Summary updated