Classification of Discourse Coherence Relations: An Exploratory Study using Multiple Knowledge Sources

Ben Wellner, James Pustejovsky, Catherine Havasi, Anna Rumshisky, Roser SauríSIGDIAL Workshop 2006 · OpenReview

Abstract

This study explores the identification and classification of discourse coherence relations using a feature-rich Maximum Entropy classifier. By leveraging a variety of linguistic sources—including syntax, temporal ordering, and deep lexical semantics—the authors demonstrate high accuracy on the Discourse GraphBank corpus, which models discourse as a graph rather than a rigid tree structure. The approach achieves 81% accuracy in relation classification, proving that a "flat" feature-based methodology is highly effective for modeling non-tree discourse structures.

Introduction

In Natural Language Processing (NLP), understanding the relationships between text segments is a complex task. Traditional models often represent discourse as a tree, where each segment has a single parent. However, real-world discourse is often non-linear, featuring overlapping relations and crossing dependencies. This paper addresses two primary sub-tasks: relation identification (detecting if a relation exists) and relation classification (determining the type of relation, such as elaboration or contrast).

Methodology

The authors employ a "flat," feature-based approach using a Maximum Entropy (MaxEnt) classifier. This model integrates a diverse set of linguistic features extracted from specialized tools to capture the "hints" necessary for classification:

  • Syntactic Features: POS tagging and dependency parsing via the RASP parser.
  • Modality (SlinkET): Identifying whether events are factual, possible, or evidential.
  • Temporal Relations (EvITA): Determining the chronological ordering of events.
  • Lexical Semantics: Utilizing the Word Sketch Engine (WSE) for distributional similarity and the Brandeis Semantic Ontology (BSO) for deep semantic paths based on Generative Lexicon theory.
Table 1: Definitions and examples of the various feature classes used in the classification model, ranging from basic proximity to deep semantic paths.

Dataset: The Discourse GraphBank

The study utilizes the Discourse GraphBank, which differs from traditional corpora like the RST Treebank by allowing for cycles and crossing dependencies. The dataset includes 135 news articles with 11 types of coherence relations, which the authors also group into four coarse-grained classes: Resemblance, Cause-Effect, Temporal, and Attribution.

Key Results

The MaxEnt classifier achieved an overall accuracy of 81.06% for fine-grained relation classification, significantly outperforming the majority-class baseline of 45.7%. When evaluated on coarse-grained categories, accuracy improved to 87.51%.

Table 2: Performance metrics (Precision, Recall, F-measure) for individual discourse relations. Note the high performance for common relations like elaboration (elab) versus the struggle with sparse data like generalization (gen).

Ablation studies revealed that "cue words" and "proximity" are the most influential features. Removing cue words resulted in a ~5% drop in accuracy, while removing proximity features led to a ~10% decrease. Furthermore, the inclusion of inter-relation dependencies—features that model the other relations a segment is involved in—boosted accuracy to 82.3%.

Discussion and Future Work

While the model performs well, several challenges remain. Implicit relations, which lack explicit cue words, are difficult to classify and may require better anaphora resolution. Additionally, the utility of the Brandeis Semantic Ontology was limited by the absence of Word Sense Disambiguation (WSD); noise in semantic paths occurs when word senses are conflated. Future work aims to integrate these components into a full discourse parser capable of modeling the entire structure of a document simultaneously.

Reference: Ben Wellner, James Pustejovsky, Catherine Havasi, Anna Rumshisky, Roser Saurí. "Classification of Discourse Coherence Relations: An Exploratory Study using Multiple Knowledge Sources." Proceedings of the 7th SIGdial Workshop on Discourse and Dialogue. 2006.

Summary updated

BibTeX
@inproceedings{wellner2006classification,
  author = {Ben Wellner and James Pustejovsky and Catherine Havasi and Anna Rumshisky and Roser Saurí},
  title = {{Classification of Discourse Coherence Relations: An Exploratory Study using Multiple Knowledge Sources}},
  booktitle = {SIGDIAL Workshop},
  year = {2006},
  url = {https://openreview.net/forum?id=xymSG9-p0X},
}
Figure