Abstract
This study explores the identification and classification of discourse coherence relations using a feature-rich Maximum Entropy classifier. By leveraging a variety of linguistic sources—including syntax, temporal ordering, and deep lexical semantics—the authors demonstrate high accuracy on the Discourse GraphBank corpus, which models discourse as a graph rather than a rigid tree structure. The approach achieves 81% accuracy in relation classification, proving that a "flat" feature-based methodology is highly effective for modeling non-tree discourse structures.
Introduction
In Natural Language Processing (NLP), understanding the relationships between text segments is a complex task. Traditional models often represent discourse as a tree, where each segment has a single parent. However, real-world discourse is often non-linear, featuring overlapping relations and crossing dependencies. This paper addresses two primary sub-tasks: relation identification (detecting if a relation exists) and relation classification (determining the type of relation, such as elaboration or contrast).
Methodology
The authors employ a "flat," feature-based approach using a Maximum Entropy (MaxEnt) classifier. This model integrates a diverse set of linguistic features extracted from specialized tools to capture the "hints" necessary for classification:
- Syntactic Features: POS tagging and dependency parsing via the RASP parser.
- Modality (SlinkET): Identifying whether events are factual, possible, or evidential.
- Temporal Relations (EvITA): Determining the chronological ordering of events.
- Lexical Semantics: Utilizing the Word Sketch Engine (WSE) for distributional similarity and the Brandeis Semantic Ontology (BSO) for deep semantic paths based on Generative Lexicon theory.
Dataset: The Discourse GraphBank
The study utilizes the Discourse GraphBank, which differs from traditional corpora like the RST Treebank by allowing for cycles and crossing dependencies. The dataset includes 135 news articles with 11 types of coherence relations, which the authors also group into four coarse-grained classes: Resemblance, Cause-Effect, Temporal, and Attribution.
Key Results
The MaxEnt classifier achieved an overall accuracy of 81.06% for fine-grained relation classification, significantly outperforming the majority-class baseline of 45.7%. When evaluated on coarse-grained categories, accuracy improved to 87.51%.
Ablation studies revealed that "cue words" and "proximity" are the most influential features. Removing cue words resulted in a ~5% drop in accuracy, while removing proximity features led to a ~10% decrease. Furthermore, the inclusion of inter-relation dependencies—features that model the other relations a segment is involved in—boosted accuracy to 82.3%.
Discussion and Future Work
While the model performs well, several challenges remain. Implicit relations, which lack explicit cue words, are difficult to classify and may require better anaphora resolution. Additionally, the utility of the Brandeis Semantic Ontology was limited by the absence of Word Sense Disambiguation (WSD); noise in semantic paths occurs when word senses are conflated. Future work aims to integrate these components into a full discourse parser capable of modeling the entire structure of a document simultaneously.
Reference: Ben Wellner, James Pustejovsky, Catherine Havasi, Anna Rumshisky, Roser Saurí. "Classification of Discourse Coherence Relations: An Exploratory Study using Multiple Knowledge Sources." Proceedings of the 7th SIGdial Workshop on Discourse and Dialogue. 2006.
Summary updated