TL;DR
This paper investigates whether automatically generated discourse structure features can improve the prediction of argumentation structure in text. By leveraging existing discourse parsers (RST and PDTB) on a specialized corpus, they demonstrate that these features, particularly from PDTB, significantly enhance argumentation parsing performance, especially for classifying argument function and attachment.
What Problem Does This Paper Address?
Argumentation mining aims to automatically identify claims and their supporting or attacking arguments within a text. This is closely related to discourse parsing, which analyzes how sentences and clauses connect to form coherent text. While previous work has shown that gold-standard discourse annotations (like Rhetorical Structure Theory - RST) can help predict argumentation structure, there's been a lack of empirical studies using automatically parsed discourse features. This paper bridges that gap by exploring whether off-the-shelf discourse parsers can provide useful signals for argumentation mining, moving towards a more practical, real-world scenario.
Methods
The authors use a corpus of "argumentative microtexts" that has been annotated for both argumentation structure and discourse structure (RST and Segmented Discourse Representation Theory - SDRT). They employ existing RST and Penn Discourse Treebank (PDTB) parsers to generate discourse structures for these texts.
Key methodological steps include:
- Parser Selection: They evaluate several RST and PDTB parsers and select the most promising ones for their experiments.
- Data Preparation: To overcome issues with parser-generated segment boundaries not matching the gold annotations, they use pre-segmented text from the gold corpus as input for the parsers. This is noted as a simplification of a "real-world" scenario.
- Feature Engineering: They extract features from the output of the discourse parsers. These include information about RST relations and PDTB connectives and their relations.
- Argumentation Parsing Model: They replicate a state-of-the-art argumentation parsing model (Afantenos et al., 2018), which uses an "evidence graph" approach with an MST decoder. They then augment this model with the extracted discourse features.
- Evaluation: They compare the performance of the argumentation parser with and without the discourse features, using macro-averaged F1 scores across different classifiers (central claim, role, function, attachment).
Datasets
- Argumentative Microtext Corpus: This is the primary dataset used. It consists of 112 short texts, originally in German and professionally translated to English. The corpus is annotated for both argumentation structure (claim, support, attack) and discourse structure (RST and SDRT). This corpus is unique in offering parallel annotations for both domains.
Key Insights and Ideas
- Discourse Structure is Informative for Argumentation: The core finding is that automatically derived discourse structure features can indeed improve argumentation parsing. This validates the intuition that how a text is structured linguistically is linked to how arguments are presented.
- PDTB Features Show Strong Potential: While RST features have been explored before, the PDTB features proved particularly beneficial, especially for classifying the function of an argument (e.g., support, rebuttal) and its attachment point (which other argument it relates to).
- Automatic Parsers are Imperfect but Useful: The study acknowledges that automatic discourse parsers don't perfectly match gold annotations (e.g., segment boundaries differ). However, even with these imperfections, the extracted features still provide valuable signals.
- Feature Combination is Key: Combining RST and PDTB features, along with existing lexical/syntactic features, yielded the best overall performance, suggesting a synergistic effect.
Key Results
- Improved Argumentation Parsing: The addition of PDTB features to the baseline argumentation parsing model led to significant improvements, particularly for the function (fu) and attachment (at) classifiers.
- PDTB vs. RST: While both RST and PDTB features offered some improvement, PDTB features generally showed a larger impact. For example, the "Default, PDTB" model achieved a macro-averaged F1 score of 0.420 for function, compared to 0.278 for "Default, RST."
- Best Performance: The model incorporating "Default, 2018, RST, RST+, PDTB" features achieved the highest F1 scores, reaching 0.854 for central claim (cc), 0.737 for role (ro), 0.461 for function (fu), and 0.732 for attachment (at). The highest score for function (0.461) was achieved with the combination of all features for the simple relation set.
- Central Claim Matching: The automatic RST parser matched the "most nuclear" segment (a proxy for the central claim) in about 60% of texts, which is considered a promising score given the complexity.
Limitations and Future Work
- Pre-segmented Text: A key limitation acknowledged is the use of pre-segmented text as input to the parsers. This simplifies the task by removing the challenge of jointly parsing discourse and argumentation structures with potentially misaligned segments. The authors note this is a simplification of the "real world" scenario.
- PDTB Coverage: The PDTB parser identified a relatively small number of relations, and these didn't always map well to the argumentation annotations.
- Future Work:
- Conduct a qualitative error analysis of how parser contributions affect argumentation parsing.
- Perform an ablation study to pinpoint the impact of individual RST and PDTB features.
- Apply similar experiments to a newly released, larger part of the microtext corpus.
- Investigate the potential influence of the German-to-English translation step on the results.
Summary updated