Argument Component Classification by Relation Identification by Neural Network and TextRank

Mamoru Deguchi, Kazunori YamaguchiACL 2019

TL;DR

This paper introduces a novel method for argument mining that combines neural networks for relation identification with the TextRank algorithm to improve argument component classification. By leveraging the strength of relationships between argument components, the model can better identify claims and premises, outperforming previous methods on student essays and showing promise for broader applications.

What Problem Does This Paper Address?

Argumentation mining, the task of automatically extracting the structure of arguments from text, is a growing field. A crucial sub-task is argument component classification, where individual pieces of text are labeled as "claims" (conclusions) or "premises" (reasons supporting claims). Existing methods fall into two camps: supervised methods that often focus on single sentences and miss broader document context, and unsupervised methods that can use the whole document but lack accuracy. This paper aims to bridge this gap by developing a method that utilizes document-level context and argumentation-specific knowledge without requiring extensive manual feature engineering for new domains.

Methods

The core of the proposed method lies in a two-stage process:

  1. Relation Identification using a Neural Network: The authors employ a neural network to determine the probability of a relationship (a "link") between any two argument components (ACs). This network first converts each AC into a sentence vector using either LSTM or CNN. Then, another network (specifically, an LSTM followed by dense layers) takes these vectors and outputs the probability of a relation. This allows the model to learn argumentation-specific knowledge from a small corpus.
  2. Argument Component Classification using TextRank: The probabilities generated by the neural network are then used as weights in the TextRank algorithm. TextRank, a graph-based ranking algorithm, is adapted here to rank argument components. Components with a higher probability of being related to other components (especially those likely to be claims) are ranked higher. This effectively integrates the learned relation information into the classification process, allowing the model to leverage document-wide context.

The novelty lies in using the neural network's relation probabilities as weights for TextRank, thereby combining the strengths of both approaches.

Datasets

The authors evaluated their method on two distinct corpora:

  • Student Essay: This corpus, sourced from the online forum essayforum.com, consists of annotated student essays in English. Argument components are classified as major claims, claims, or premises. Relations between ACs are also annotated.
  • Wikipedia Article: This corpus, derived from annotated Wikipedia articles, includes topic labels, Context Dependent Claims (CDCs, statements supporting or attacking the topic), and Context Dependent Evidence (CDEs). For this study, CDCs and related CDEs were considered as positive examples of relations.

Key Insights and Ideas

  • Leveraging Relations is Key: The paper strongly suggests that understanding the relationships between argument components is crucial for accurate classification, especially for identifying claims. A component's role is often defined by its connection to others.
  • Bridging Supervised and Unsupervised: The proposed method ingeniously combines the contextual understanding of unsupervised methods (like TextRank) with the learned knowledge of supervised methods (the neural network for relation identification).
  • Domain Adaptability: By learning relation patterns from a small corpus, the method aims to be more adaptable to new argumentation domains with less manual feature engineering, a significant advantage over many existing techniques.
  • Ranking vs. Classification: The paper frames component classification as a ranking problem, which can be more nuanced than a strict binary or multi-class classification, particularly when dealing with varying numbers of ACs.

Key Results

  • Student Essay Performance: The proposed method significantly outperformed TextRank-TFIDF and TextRank-W2V for identifying major claims, achieving an accuracy of 0.542 compared to 0.289 for TextRank-TFIDF. It also showed better separation of ranks for major claims, claims, and premises.
  • Wikipedia Article Performance: While the neural network classifiers achieved better F-scores on the Wikipedia Article dataset, the proposed method demonstrated higher precision for identifying claims at rank 1 (@1). This suggests its utility when precise identification of key claims is paramount.
  • Comparison to Neural Classifiers: For student essays, the proposed method was superior in F-score for major claims. For Wikipedia articles, it was competitive, offering better precision in certain scenarios.

Limitations and Future Work

  • Wikipedia Article F-Score: The authors acknowledge that for the Wikipedia Article dataset, their method was slightly worse than neural network classifiers in terms of F-score.
  • Future Directions: The authors plan to explore using word vectors that capture contextual information to further improve relation identification. They also intend to test other ranking algorithms, such as RankNet and ListNet, to see if they can further enhance the performance.

Summary updated

BibTeX
@inproceedings{deguchi2019argument,
  author = {Mamoru Deguchi and Kazunori Yamaguchi},
  title = {{Argument Component Classification by Relation Identification by Neural Network and TextRank}},
  booktitle = {Annual Meeting of the Association for Computational Linguistics},
  year = {2019},
}
Figure