Relational Summarization for Corpus Analysis

Abram Handler, Brendan T. O'ConnorNAACL 2018 · DOI

Introduction

In the field of corpus analysis, visual tools such as concept maps and network diagrams are frequently employed to navigate large volumes of text. While these tools effectively highlight connections between entities, they often fail to explain the underlying nature of those connections. When an analyst observes a link between two terms, the immediate question is: "How are these two related?"

Figure 1: A concept map interface where selecting a relationship between "Aristide" and "Liberation Theology" provides explanatory snippets from the corpus.

Existing solutions, such as Open Information Extraction (OpenIE) or standard document summarization, are often ill-suited for this task. OpenIE focuses on extracting structured (subject, relation, object) triples for database population, frequently discarding the specific, "messy" details that are most valuable to a human analyst. Conversely, standard summarization focuses on broad themes rather than the specific intersection of two arbitrary terms. This paper proposes relational summarization, a task designed to generate readable, natural language descriptions of term relationships directly from a text corpus.

Methodology

The authors approach relational summarization as a specialized form of supervised sentence compression. The process is divided into two primary stages: candidate set generation and summary construction.

Figure 4: The relational summarization pipeline, transitioning from a raw mention set to a filtered candidate set and finally a concise summary.

Candidate Set Generation

The system first identifies the "mention set"—all sentences within the corpus containing both target terms ($t_1$ and $t_2$). Rather than extracting a predicate, the model attempts to "compress" these sentences into coherent snippets that begin with $t_1$ and end with $t_2$. A logistic regression classifier, trained on a gold-standard sentence-compression dataset, determines the "acceptability" of potential spans. This classifier utilizes grammatical features, including Part-of-Speech (POS) tags and dependency edge labels, to ensure the resulting snippet is human-readable.

Summary Construction

Once a set of readable candidates is generated, the system must select the most representative statements to present to the user. This stage involves balancing information density with the need to avoid redundancy, particularly in heterogeneous corpora where relationships may evolve over time or involve multiple distinct events.

Experimental Evaluation

The model was evaluated using two diverse datasets: a collection of New York Times articles (1987–2007) focusing on Haiti, and a large-scale crawl of the "relationships" subreddit on Reddit (2015–2017).

Key Results

  • Increased Yield: The proposed method significantly outperformed traditional tools in terms of coverage. In the Haiti corpus, it generated candidate sets for 739 entity pairs, compared to 443 for Stanford OpenIE and just 128 for ClausIE.
  • Superior Readability: Human evaluations conducted via crowdsourcing indicated that the system's output was more readable than baseline OpenIE systems. On a 1-5 scale, the method achieved mean acceptability scores between 3.85 and 3.94, whereas Stanford OpenIE's performance dropped significantly on informal social media data.
  • Robustness: The reliance on syntactic features allowed the model to maintain high performance across both formal journalistic prose and informal online discourse.

Conclusion and Future Work

Relational summarization prioritizes the needs of human analysts by providing detailed, corpus-specific context that structured extraction methods often omit. While the current intra-sentence approach provides high yield and readability, future work aims to address multi-sentence relationships and improve the temporal awareness of the summary construction phase to better distinguish between transient events and long-term state changes.

Summary updated

BibTeX
@inproceedings{handler2018relational,
  author = {Abram Handler and Brendan T. O'Connor},
  title = {{Relational Summarization for Corpus Analysis}},
  booktitle = {Conference of the North American Chapter of the Association for Computational Linguistics},
  year = {2018},
  doi = {10.18653/v1/n18-1159},
}
Figure