TL;DR
This paper tackles the challenge of automatically understanding how paragraphs in a text relate to each other to form larger discourse structures. They propose a novel approach that combines hierarchical clustering with a machine learning model trained on a rich set of features, outperforming simpler methods based solely on word co-occurrence.
What Problem Does This Paper Address?
Understanding how sentences and paragraphs connect to form a coherent text is crucial for many Natural Language Processing (NLP) tasks like summarization and question answering. While identifying relationships between individual sentences (low-level discourse structure) has seen progress, inferring these relationships at a higher level, between paragraphs, is significantly harder. This is because traditional methods heavily relying on explicit "cue phrases" (like "however" or "therefore") become less effective and informative at the paragraph level. The paper aims to automatically predict this inter-paragraph structure, specifically focusing on determining how paragraphs attach to each other to form a hierarchical tree-like representation of the text.
Methods
The core of the proposed method involves two main components:
- Hierarchical Agglomerative Clustering: This is used as the primary mechanism for building the discourse tree. It starts with individual paragraphs as segments and iteratively merges adjacent segments until the entire text is represented as a single segment (the root of the tree). This process naturally creates a binary tree structure where leaves are paragraphs.
- Machine Learning Model for Segment Relatedness: Instead of relying on a simple similarity measure (like word co-occurrence, as in previous work), the authors train a supervised machine learning model to assess the likelihood that two adjacent segments should be merged. This model integrates a diverse set of features, including:
- Segment Position: Where the segments appear in the text (beginning, middle, end).
- Segment Length: Number of words, sentences, and paragraphs.
- Term Overlap: Based on word co-occurrence.
- Punctuation: Presence of quotation marks.
- Tense: Tense of verbs within segments.
- Cue Phrases: Explicit discourse markers.
- Lexical Chains: Sequences of semantically related words, indicating subtopics.
A maximum entropy learner is used to train this model, assigning weights to each feature based on its predictive power.
Datasets
The primary dataset used is the RST Discourse Treebank (RST-DT). This corpus contains 385 Wall Street Journal articles manually annotated with discourse structure according to Rhetorical Structure Theory (RST). For their experiments, the authors used 342 texts for training and 43 for testing. They also utilized a subset of 52 texts that were doubly annotated to estimate human agreement. The paper focuses specifically on inter-paragraph structure, discarding intra-paragraph annotations.
Key Insights and Ideas
- Rich Feature Sets are Key: The paper strongly suggests that relying on a single feature like word co-occurrence is insufficient for high-level discourse structure prediction. Combining a variety of linguistic and structural cues (tense, cue phrases, lexical chains, position, length) significantly improves performance.
- Lexical Chains are Highly Informative: Removing lexical chain features resulted in a greater performance drop than removing term overlap features, indicating their strong contribution to understanding segment relatedness.
- Paragraph Boundaries are Not Arbitrary: The research supports the intuition that paragraph breaks often align with discourse segment boundaries, making them a useful starting point for inferring higher-level structure.
- Left-Branching Structures are Common: Interestingly, the experiments found that left-branching trees (where segments tend to merge with their left neighbor) performed better than right-branching trees, contradicting some prior assumptions.
Key Results
The proposed model, combining hierarchical clustering with the machine learning model (ME), achieved the best performance across precision, recall, and F-score on the RST-DT test set.
- The F-score for the ME model was 59.00%, significantly outperforming the word co-occurrence baseline (TO) at 50.79% and left-branching trees (LB) at 54.27%.
- Removing lexical chains (ME-LC) reduced performance to 55.86%, while removing term overlap (ME-TO) resulted in 57.12%.
- Even when both lexical chains and term overlap were removed (ME-LCTO), the model still performed better than left-branching trees, achieving 56.66%.
- Human performance on the doubly annotated set was considerably higher, with an F-score of 64.34%, indicating room for improvement in automatic methods.
Limitations and Future Work
- Rhetorical Relation Prediction: The current work focuses on predicting the structure (how segments attach). The authors plan to extend their approach to predict the type of rhetorical relation (e.g., Elaboration, Contrast) between these segments in future work. They note that predicting relations at the inter-paragraph level is easier than at the sentence level due to less variation.
- Relaxing Tree Constraint: The current model assumes a tree structure with non-crossing branches. The authors suggest investigating whether relaxing this constraint could lead to better representations, as some researchers propose.
- Domain Generalization: The study was conducted on Wall Street Journal articles. Future work should explore whether these methods generalize to other domains, especially with the availability of larger discourse-annotated corpora like the Penn Discourse Treebank.
Summary updated