Improving LSA-based Summarization with Anaphora Resolution

Josef Steinberger, Mijail Kabadjov, Massimo Poesio, Olivia Sanchez-GrailletEMNLP 2005

TL;DR

This paper introduces a novel approach to text summarization by integrating anaphora resolution with Latent Semantic Analysis (LSA). They demonstrate that by using an anaphora resolver to enrich the document representation fed into LSA, summarization performance can be significantly improved, outperforming a purely lexical LSA approach. Crucially, the way anaphoric information is integrated matters, with a direct "addition" method yielding better results than a simple "substitution" of terms.

What Problem Does This Paper Address?

Traditional term-based summarization methods, including those using Latent Semantic Analysis (LSA), often rely solely on lexical information to identify important terms and extract relevant sentences. However, a significant portion of a document's meaning is conveyed through anaphora – pronouns, definite descriptions, and other referring expressions that point to entities already mentioned. These anaphoric expressions often lack direct lexical similarity with their antecedents, making them difficult for purely lexical methods to capture. This gap means that important entities and their relationships might be overlooked, potentially leading to less comprehensive and accurate summaries. The authors aim to bridge this gap by exploring how to effectively incorporate anaphoric information into LSA-based summarization.

Methods

The core of the paper lies in enhancing an existing LSA-based summarization framework. The standard LSA approach involves creating a term-by-sentence matrix, applying Singular Value Decomposition (SVD) to uncover latent semantic topics, and then selecting sentences that best represent these topics.

The authors introduce two methods for integrating anaphoric information, obtained from an anaphora resolution system called GUITAR:

  • Substitution Method: This is a preprocessing step where anaphoric expressions in the text are replaced by their antecedents. The modified text is then used to build the SVD input matrix. The idea is to make the lexical representation more explicit.
  • Addition Method: This approach modifies the SVD input matrix directly. Instead of just representing lexical terms, the matrix also includes "chain terms" that represent anaphoric chains. This allows the system to explicitly consider sentences that mention entities belonging to the same anaphoric chain, alongside lexical terms. This method also allows for assigning potentially higher weights to anaphoric information.

Datasets

The primary dataset used for evaluation is the CAST corpus. This corpus consists of news articles from the Reuters Corpus and popular science texts from the British National Corpus, along with manually produced summaries. The corpus provides annotations for sentence importance (essential, important, linked), which are used to create reference summaries for evaluation. The authors also mention using a manually annotated subset of the CAST corpus for establishing an "upper bound" on performance with perfect anaphora resolution.

Key Insights and Ideas

  • Anaphora Resolution is Crucial for Summarization: The paper strongly suggests that ignoring anaphora leads to suboptimal summarization performance. Effectively incorporating this information can significantly boost summary quality.
  • The "How" Matters More Than the "What": Simply substituting anaphoric terms with their antecedents (Substitution Method) did not yield significant improvements. This highlights that the structure of how anaphoric information is integrated into the summarization model is critical.
  • Explicitly Modeling Discourse Entities: The "Addition Method," which creates specific representations for anaphoric chains, appears to be more effective. This suggests that explicitly modeling discourse entities, rather than just their lexical manifestations, is beneficial.
  • Automatic Resolvers Can Be Effective: The study demonstrates that even with an automatic anaphora resolver (GUITAR) that has reasonable performance, significant summarization improvements can be achieved, not just with "perfect" anaphora resolution.

Key Results

The paper presents quantitative results using several evaluation metrics: Relative Utility (RU), F-score, Cosine Similarity, and Main Topic Similarity.

  • Significant Improvements with Addition Method: Both the 15% and 30% summarization ratio evaluations show that the "Addition Method" using GUITAR consistently outperformed the baseline LSA summarizer and the "Substitution Method." For instance, at a 30% summarization ratio, Relative Utility increased from 0.810 (baseline) to 0.868 (Addition Method), a significant improvement.
  • Substitution Method Underperforms: The "Substitution Method" showed little to no significant improvement over the baseline, even when tested with "perfect" anaphora resolution.
  • GUITAR Performance: The GUITAR anaphora resolver achieved a precision of 56% and a recall of 51% on the evaluated corpus.

Limitations and Future Work

  • Limited Scope of Anaphora Resolution: The GUITAR system used in this study did not include methods for resolving proper nouns. The authors suggest that extending GUITAR with proper noun resolution would be a valuable future direction.
  • Dataset Specificity: While the CAST corpus is a good benchmark, the authors acknowledge the need to extend the set of documents and test the system with other collections to ensure generalizability.
  • Further Exploration of "Addition" Method: While the "Addition Method" showed promise, further work could explore different weighting schemes for anaphoric terms within this framework.
  • Evaluating Other Anaphora Resolution Components: The paper mentions that the contribution of different anaphora resolution components is addressed in a separate work (Kabadjov et al., 2005), implying that a deeper dive into which specific types of anaphora are most impactful could be a future avenue.

Summary updated

BibTeX
@inproceedings{steinberger2005improving,
  author = {Josef Steinberger and Mijail Kabadjov and Massimo Poesio and Olivia Sanchez-Graillet},
  title = {{Improving LSA-based Summarization with Anaphora Resolution}},
  booktitle = {Conference on Empirical Methods in Natural Language Processing},
  year = {2005},
}
Figure