TL;DR
This paper tackles the challenge of automatically identifying the "arguments" (the pieces of text that a discourse connective like "because" or "however" relates) within a document. Instead of finding the full text spans, they focus on identifying the "head words" of these arguments, simplifying the problem and achieving over 74% accuracy by combining dependency parsing features with a re-ranking model.
What Problem Does This Paper Address?
Understanding how sentences and clauses relate to each other in a larger text (discourse) is crucial for many NLP tasks like summarization and question answering. Discourse connectives (e.g., "and," "because," "nevertheless") explicitly signal these relationships. However, automatically identifying what text segments these connectives are relating (their arguments) is difficult. Existing approaches often struggle with segmenting the text into the correct units and then determining which units form the arguments. This paper focuses on a specific challenge: identifying the core word (head word) that represents each argument, which is a more manageable sub-problem than identifying the entire argument span.
Methods
The authors propose a two-stage approach:
- Independent Argument Identification: They first build models to identify ARG1 and ARG2 (the two arguments of a connective) independently. They experiment with various features derived from:
- Baseline Features: Basic information about the connective and argument position.
- Constituent Path Features: Features derived from traditional phrase-structure parse trees.
- Dependency Path Features: Features derived from dependency parse trees, which represent grammatical relationships between words. The authors found these to be more effective.
- Connective Features: Information about the type of connective (coordinating, subordinating, adverbial).
-
Lexico-Syntactic Features: Features related to attribution and verb complements.
They use a log-linear ranking model for this stage, where candidate arguments are ranked against each other.
-
Inter-Argument Dependency Modeling (Re-ranking): To improve accuracy, they then model the dependencies between the two arguments. They use a re-ranking model that takes the top N candidate argument pairs identified in the first stage and re-ranks them based on features that consider both arguments simultaneously. This allows them to capture relationships like the relative order of arguments and connectives within sentences.
Datasets
- Penn Discourse TreeBank (PDTB): This is the primary dataset used for both training and evaluation. The PDTB annotates discourse relations signaled by explicit connectives and their arguments. The authors use sections 02-22 for training, 00-01 for development, and 23-24 for testing.
Key Insights and Ideas
- Head Word Identification is Sufficient: The paper argues that identifying the "head word" of a discourse argument is often sufficient and preferable to identifying the full argument extent, simplifying the problem and bypassing difficult discourse segmentation issues.
- Dependency Parses Outperform Constituent Parses: Features derived from dependency parse trees yielded significantly better results than those from constituent parse trees for argument identification. This suggests that dependency structures offer a more compact and informative representation for this task.
- Re-ranking Improves Accuracy: Modeling inter-argument dependencies through a re-ranking model, after initial independent identification, leads to a notable improvement in overall accuracy. This highlights the importance of considering how the two arguments relate to each other.
- ARG2 Identification is Easier: The accuracy for identifying ARG2 (the argument syntactically connected to the connective) is significantly higher than for ARG1, which can appear much earlier in the discourse. This aligns with human annotation agreement rates.
Key Results
- Overall Accuracy: The system achieves 74.2% accuracy in correctly identifying both arguments of a connective using gold-standard parses and the full feature set (A-Y).
- ARG1 vs. ARG2 Accuracy: The system achieves 76.4% accuracy for ARG1 and 95.4% accuracy for ARG2 on the test set with gold-standard parses.
- Connective Accuracy: 74.2% connective accuracy (both arguments correctly identified) is achieved with gold-standard parses.
- Automatic Parses: Using automatic parses (Charniak-Johnson parser), the system achieves 69.8% ARG1 accuracy, 90.8% ARG2 accuracy, and 64.6% connective accuracy.
- Dependency vs. Constituent Features: The dependency parse features (A-G;M-Q) achieved 70.2% connective accuracy, significantly outperforming constituent parse features (A-L) at 53.6%.
Limitations and Future Work
- ARG1 Identification Challenges: The paper acknowledges that identifying ARG1s, especially for anaphoric discourse adverbials, remains challenging. This is partly due to the difficulty in capturing lexico-semantic and attributive properties.
- Connective Type Specificity: The authors suggest that separate models and feature sets for different connective types (coordinating, subordinating, adverbial) might yield further improvements.
- Inter-Connective Dependencies: While they model inter-argument dependencies for a single connective, they hypothesize that modeling dependencies between different connectives and their arguments (e.g., avoiding crossing argument links) could lead to even greater gains.
- Jointly Modeling PropBank and PDTB: The authors suggest that jointly modeling semantic role labeling (PropBank) and discourse relations (PDTB) is a promising future direction, for which their head-based and dependency-based approach would be well-suited.
Summary updated