Abstract
Researchers from Shanghai Jiao Tong University developed a pipeline-based system for shallow discourse parsing that treats the problem as a sequence of classification subtasks. By leveraging constituent parsing trees and Maximum Entropy models, the system identifies both explicit and implicit discourse relations. While the system performs well on connective detection, it highlights the significant difficulty of accurately extracting argument spans in natural language.
Introduction
Discourse parsing is the fundamental task of uncovering the logical structure of a text by identifying how sentences and clauses relate to one another. For instance, determining whether a sentence provides a reason for a previous statement or presents a contrasting viewpoint. This paper focuses on "Shallow Discourse Parsing" as defined by the CoNLL-2015 Shared Task, which involves identifying discourse connectives (explicit words like "but" or "because"), non-explicit relations (implied links), the arguments (Arg1 and Arg2) being linked, and the specific sense of the relationship.
System Architecture
The authors designed a sequential pipeline that mimics the manual annotation process used in the Penn Discourse Treebank (PDTB). The system is bifurcated into two main branches: the Explicit Pipeline and the Non-explicit Pipeline.
1. The Explicit Pipeline
- Explicit Classifier: This component utilizes a level-order traversal of a constituent parse tree to identify potential connectives. It relies on syntactic features such as the "Self Category" (the node covering the connective) and its siblings.
- Argument Labeler: Once a connective is identified, the system examines all nodes branching off the path from the connective to the root of the tree to find candidates for Arg1 or Arg2.
- Sense Classifier: A Maximum Entropy (MaxEnt) classifier assigns a relationship meaning based on the connective and its immediate context.
2. The Non-explicit Pipeline
- Filter: This stage removes cases that are clearly not discourse relations, such as those already identified as explicit.
- Non-explicit Classifier: Categorizes the remaining pairs into "Implicit," "AltLex" (Alternative Lexicalization), or "EntRel" (Entity Relation).
- Sense Classifier: For implicit relations where no explicit signal exists, the authors employ "word-pair" features (e.g., the first and last words of each argument) to infer the semantic relationship.
Experimental Setup
The system was trained and evaluated using the Penn Discourse Treebank (PDTB), the gold standard for discourse analysis. The evaluation utilized the CoNLL-2015 Shared Task datasets, including training, development, test, and blind test sets. The "closed track" constraints of the task meant the system could only use provided resources, emphasizing the importance of algorithmic design and feature engineering.
Results and Discussion
The system's performance varied significantly across subtasks, reflecting the inherent complexity of discourse parsing. While connective detection was highly successful, argument extraction remained a major bottleneck.
Key performance metrics from the blind test set include:
- Connective Detection: Achieved a strong F1-score of 0.8168.
- Argument Extraction: Identifying the exact spans for both Arg1 and Arg2 proved extremely difficult, resulting in a low F1-score of approximately 0.047.
- Overall Parser Performance: The final F1-score for the complete pipeline (correctly identifying connectives, arguments, and senses) was 0.0358.
Conclusion
The study demonstrates that while syntactic trees are powerful tools for identifying connectives and pruning search spaces, they are currently insufficient for precise argument span extraction. The sequential nature of the pipeline leads to error propagation, where failures in early stages (like argument labeling) significantly degrade the final parser score. Future work will focus on refining the Argument Labeler, potentially through more sophisticated tree navigation or alternative machine learning architectures to better define text boundaries.
Summary updated