TL;DR
This paper presents a shallow discourse parser for identifying semantic relationships between text segments. Their system, submitted to the CoNLL-2016 Shared Task, leverages a pipeline of data-driven approaches and introduces novel semantic features for the challenging task of implicit relation detection, showing promising improvements.
What Problem Does This Paper Address?
Discourse parsing aims to uncover the underlying structure and meaning of text by identifying how different parts of a discourse relate to each other. These relationships can be explicit (signaled by words like "because" or "however") or implicit (where the connection is implied by meaning). While explicit discourse parsing has seen progress, implicitly signaled relations remain a significant challenge due to the lack of direct linguistic cues. This paper tackles the problem of building a robust shallow discourse parser that can effectively handle both explicit and, crucially, implicit discourse relations, contributing to a deeper understanding of text coherence and meaning.
Methods
The authors propose a pipeline-based approach for their Shallow Discourse Parser (SDP). The pipeline is divided into two main branches: one for explicit relations and another for non-explicit (implicit) relations.
For explicit discourse parsing, the pipeline involves:
- Connective Detection: Identifying words or phrases that signal a discourse relation. They use a Maximum Entropy Classifier, incorporating syntactic features, to distinguish discourse connectives from non-discourse uses of the same words.
- Argument Labeling: Locating the text spans (arguments) that are related by the connective. This involves classifying argument positions (same sentence vs. previous sentence) and then extracting argument spans using a modified version of Kong's constituency-based approach, which is adapted for multi-word connectives.
- Argument Classification: Assigning the extracted spans to Arg1, Arg2, or Null.
- Explicit Sense Classification: Determining the specific type of semantic relation (e.g., Cause, Contrast) using features like connective string, POS tags, and semantic lexicons like VerbNet. They found Naive Bayes to outperform MaxEnt for this task.
For non-explicit discourse parsing, the system focuses on:
- Implicit Sense Classification: This is identified as the bottleneck. They treat adjacent sentences without explicit connectives as implicit relations. Their approach involves creating novel semantic features using MPQA Subjectivity, VerbNet classes, and Word2Vec. Word2Vec is used to generate vector representations of arguments, and features like
Arg1Cluster,Arg2Cluster, andcosineDistanceare derived from these vectors.
Datasets
The primary dataset used is the Penn Discourse TreeBank (PDTB) 2.0. This serves as the gold standard for training and evaluation. The paper mentions that PDTB provides phrase structure and dependency parses, which are utilized as features. They also refer to development and test datasets provided for the CoNLL-2016 Shared Task.
Key Insights and Ideas
- Context is King for Connectives: The paper highlights that simply matching connective words isn't enough; context is crucial for determining if a word like "once" functions as a discourse connective or not.
- Multi-word Connective Handling: Adapting Kong's extraction algorithm to handle multi-word connectives significantly improved the F1 score for argument extraction, demonstrating the importance of fine-grained constituent identification.
- Semantic Lexicons for Implicit Relations: The introduction of Word2Vec-based features (
Arg1Cluster,Arg2Cluster,cosineDistance) for implicit sense classification showed a notable improvement, suggesting that capturing semantic similarity and context between arguments is key to understanding implicit relations. - Naive Bayes for Sense Classification: The finding that Naive Bayes performs better than MaxEnt for explicit sense classification, potentially due to MaxEnt overfitting, is an interesting empirical result.
Key Results
- Explicit Argument Extraction: The modified Kong's extractor with multi-word handling improved the F1 score for Arg2 by about 2.7% on the development data.
- Implicit Sense Classification: The novel Word2Vec features improved the F1 score for the implicit sense classification task by approximately 2.3% on the development dataset.
- Overall Performance: Table 2 shows their system's performance across explicit, implicit, and overall tasks on development, test, and blind datasets, comparing it with Wang's parser. While their system shows competitive results, they acknowledge areas for improvement.
Limitations and Future Work
- Naive PS-Explicit and Implicit Argument Extractors: The authors identify the "naive and hard-coded" nature of their PS-Explicit and Implicit argument extractors as a significant weak link that limits overall F1 scores. Fixing these is suggested as a major avenue for improvement.
- Error Propagation: The paper notes that error propagation from earlier stages in the pipeline can negatively impact later stages, particularly for implicit relations.
- Hard-coding Implicit Relations: Their system "hard-codes" adjacent sentences as implicit relations, which might not capture all nuances and could lead to performance dips compared to systems that more granularly identify EntRel and AltLex relations.
- Future Work: The authors suggest that further research incorporating other lexical and semantic tools could lead to more promising results in Shallow Discourse Parsing. They also imply that improving the argument extraction components would yield significant gains.
Summary updated