A More Cohesive Summarizer

Christian Smith, Henrik Danielsson, Arne JönssonCOLING 2012

TL;DR

This paper introduces COHSUM, a novel extractive summarization system that prioritizes text cohesion by leveraging coreference links between sentences. By analyzing how sentences refer to each other, COHSUM aims to produce more readable summaries, achieving competitive content coverage while significantly improving cohesion compared to existing methods.

What Problem Does This Paper Address?

Automatic text summarization, particularly extractive methods that select sentences directly from the source, often struggles with producing cohesive and readable output. While many summarizers excel at capturing the most important information (content coverage), they can result in fragmented texts with broken anaphoric references (e.g., pronouns that don't clearly refer to a noun). This lack of cohesion makes summaries difficult to understand. Existing evaluation metrics often focus on content coverage, neglecting the crucial aspect of linguistic quality and how well sentences flow together. This paper addresses this gap by proposing a summarizer specifically designed to maintain text cohesion, hypothesizing that this focus will lead to more readable summaries.

Methods

The core of COHSUM lies in its use of coreference resolution to build a graph representation of the document. Each sentence is a node, and an edge exists between two sentences if they have a coreferential relationship. The system uses the Stanford CoreNLP toolkit for coreference resolution, identifying mentions (like noun phrases, pronouns, and named entities) and their antecedents.

A key innovation is how they weight these relationships. They identify "representative mentions" within coreference chains, prioritizing proper nouns over common nouns and longer strings. Sentences containing these representative mentions are considered to have "out-links" (referring to other sentences), while sentences without them only have "in-links" (being referred to). This graph is then ranked using a variant of the PageRank algorithm, similar to TextRank. Sentences with higher ranks, indicating they are central to many coreference relationships, are selected for the summary. The summarizer extracts sentences one by one until a target word count (around 100 words) is reached, ensuring comparability with other systems.

Datasets

The primary dataset used for evaluation is the 533 news paper texts from the DUC 2002 (Document Understanding Conference) single-document summarization task. These texts were used to evaluate COHSUM against existing DUC systems and a baseline.

Key Insights and Ideas

  • Cohesion is more than just content: The paper strongly suggests that focusing solely on extracting the "most important" sentences can lead to a loss of readability due to broken coreferences.
  • Coreference chains as a structural signal: Coreference relationships provide a valuable signal for identifying sentences that are structurally important for maintaining text flow.
  • Representative mentions matter: Not all coreferences are equal. Identifying and prioritizing "representative mentions" within coreference chains helps to better weight sentence importance for cohesion.
  • PageRank for cohesion: Applying graph-based ranking algorithms like PageRank, traditionally used for link analysis, can effectively identify cohesive sentences for summarization.
  • News text structure: The study highlights that news articles often have a structure where early sentences introduce key information and later sentences elaborate. This structure naturally lends itself to cohesive summaries when coreferences are considered.

Key Results

COHSUM achieved competitive results on content coverage, performing on par with many DUC 2002 systems and a baseline (FIRST) that simply takes the first 100 words. Crucially, COHSUM significantly outperformed most other systems in terms of cohesion:

  • Intact Coreferences: COHSUM had the highest number of intact coreferences among all systems, meaning more of the original coreferential links were preserved in the summary.
  • Broken Coreferences: COHSUM had significantly fewer broken coreference chains compared to most DUC systems.
  • Comparison to FIRST: While the FIRST baseline achieved the highest number of intact coreferences and the fewest broken ones, COHSUM's performance was still strong and notably better than most other extractive summarizers in terms of cohesion.

Limitations and Future Work

The authors acknowledge that their approach to coreference weighting is simplistic, treating all coreferences as equally important regardless of type. They suggest that a more nuanced weighting scheme, considering different types of coreferences, could further improve results.

Future work includes applying COHSUM to other text types beyond news articles, such as authority texts, information texts, and academic texts, where the distribution of coreferences might differ and the importance of cohesion could be even more pronounced. They also plan to investigate how to incorporate outside word space information to enhance summarization.

Summary updated

BibTeX
@inproceedings{smith2012more,
  author = {Christian Smith and Henrik Danielsson and Arne Jönsson},
  title = {{A More Cohesive Summarizer}},
  booktitle = {International Conference on Computational Linguistics},
  year = {2012},
}
Figure