TL;DR
This paper introduces a novel probabilistic model for discourse parsing that leverages syntactic and lexical features to build sentence-level discourse trees. Their approach achieves near-human performance, significantly improving upon existing methods by effectively bridging the gap between sentence structure and discourse meaning.
What Problem Does This Paper Address?
Understanding the structure of discourse—how sentences and clauses relate to each other to form coherent arguments or narratives—is crucial for many natural language processing tasks like summarization, question answering, and machine translation. While syntactic parsing (understanding sentence structure) has seen significant advancements, discourse parsing has lagged due to a lack of large, annotated corpora. This paper tackles the challenge of automatically constructing discourse parse trees at the sentence level, aiming to achieve accuracy comparable to human annotators.
Methods
The authors break down discourse parsing into two main stages: discourse segmentation and discourse parsing.
- Discourse Segmentation: This process identifies "elementary discourse units" (edus) within a sentence. They propose a probabilistic model that uses both syntactic parse trees and lexical information to decide where to place discourse boundaries. Specifically, they analyze the head words of clauses and their syntactic context to predict these boundaries.
- Discourse Parsing: Once sentences are segmented into edus, the parser builds a discourse tree. This involves a probabilistic model that assigns probabilities to potential discourse tree structures and rhetorical relations (like "ATTRIBUTION" or "ENABLEMENT"). A key innovation here is the use of a "dominance set," derived from the segmented syntactic tree, which captures salient syntactic and lexical information at attachment points between edus. This set is used to condition the probability of discourse trees, overcoming sparseness issues. They employ a bottom-up parsing algorithm to find the most likely discourse tree.
Datasets
The primary dataset used is the RST-DT (Rhetorical Structure Theory Discourse Treebank). This corpus consists of Wall Street Journal articles annotated with discourse structures based on Rhetorical Structure Theory.
- Training: 347 articles (6132 sentences)
- Testing: 38 articles (991 sentences) A subset of 53 articles was doubly annotated by human annotators to measure inter-annotator agreement, serving as a benchmark for human-level performance.
Key Insights and Ideas
- Syntax is a Strong Indicator of Discourse: The paper empirically demonstrates a strong correlation between sentence syntax and discourse structure, suggesting that syntactic information is highly informative for discourse parsing.
- Lexical Information is Crucial for Segmentation: Simply relying on syntactic structure isn't enough for accurate discourse segmentation. The specific lexical heads of words play a vital role in determining discourse boundaries.
- Dominance Sets as a Compact Representation: The "dominance set" is a novel way to represent the crucial syntactic and lexical information from a segmented syntactic tree. This representation effectively captures the relationships between potential discourse units and is key to overcoming data sparseness in the parsing model.
- Near-Human Performance Achieved: The proposed model achieves accuracy levels that are remarkably close to human annotators, indicating a significant leap forward in automatic discourse parsing.
Key Results
- Discourse Segmentation: The proposed segmentation model achieved an error reduction of 15.1% in F-score compared to a state-of-the-art decision-based parser (Marcu, 2000). Using perfect syntactic trees further improved performance.
- Discourse Parsing: The discourse parsing model achieved an error reduction of 18.8% (F-score improvement from 37.2% to 49.0% with 18 labels) over a state-of-the-art parser.
- Near-Human Performance: With perfect syntactic trees and perfect discourse segmentation, the model's performance reached an F-score of 75.5% (18 labels) and 70.3% (110 labels), which is very close to human annotation agreement (77.0% and 71.9% respectively). This suggests the model is sophisticated enough to match near-human levels of performance.
Limitations and Future Work
- Syntactic Parser Errors: The performance of the syntactic parser, even at 90% accuracy, acts as a bottleneck. Using perfect syntactic trees significantly improves discourse parsing results.
- Discourse Segmentation Accuracy: The current discourse segmentation accuracy (83%) is identified as a limitation. Improving segmentation algorithms is seen as a promising avenue for further gains in discourse parsing.
- Model Weakness: While the model is strong, the authors acknowledge that it could be further improved.
- Future Work: The authors suggest that more effort should be directed towards improving semantic/discourse segmentation, as advancements in this area are expected to have a significant impact on both semantic and discourse parsing.
Summary updated