Context-Independent Claim Detection for Argument Mining

Marco Lippi, Paolo TorroniIJCAI 2015

Introduction

Argument mining is a critical task in natural language processing that involves extracting structured arguments—comprising premises and conclusions—from unstructured text. A fundamental challenge in this field is claim detection: the ability to identify the central statement or conclusion that an argument seeks to support. While previous systems have excelled at identifying claims related to specific, pre-defined topics, they often struggle when the topic is unknown or shifts rapidly. This paper presents a novel approach to Context-Independent Claim Detection (CD), identifying argumentative statements based solely on their linguistic and rhetorical properties.

The Challenge: Moving Beyond Context

Most state-of-the-art claim detection systems, such as IBM’s Debater project, are context-dependent. They rely on knowing the topic (e.g., "Should we ban nuclear weapons?") to find relevant claims. However, mining "in the wild"—such as on social media or across diverse web documents—requires a system that can recognize the act of making a claim regardless of the subject matter. The authors hypothesize that argumentative sentences share universal rhetorical structures that can be identified without domain-specific tuning.

Methodology: Structural Analysis with Tree Kernels

To capture these universal patterns, the researchers moved away from traditional Bag-of-Words models, which often over-rely on topic-specific keywords. Instead, they employed a structural approach using Constituency Parsing and Partial Tree Kernels (PTK).

  1. Constituency Parsing: Every sentence is transformed into a hierarchical parse tree that represents its grammatical structure, identifying components like noun phrases (NP), verb phrases (VP), and subordinate clauses (SBAR).
  2. Partial Tree Kernels: Rather than looking for exact matches, PTK measures the similarity between parse trees by identifying common sub-structures or "fragments." This allows the system to recognize when different sentences share the same underlying rhetorical "signature."
Figure 1: A constituency parse tree representing the syntactic structure of a claim sentence.

Key Insights: Rhetorical Signatures

The core finding is that claims often "look" a certain way. They frequently utilize specific verb phrases (e.g., "Critics believe that...") or comparative structures (e.g., "X is better than Y"). By using tree kernels, the model can automatically learn these patterns. For example, the system identifies common fragments involving verb phrases followed by adjective phrases, which are typical of evaluative claims.

Figure 6: Examples of structural fragments (sub-trees) that the Partial Tree Kernel identifies as characteristic of argumentative claims.

Results and Performance

The approach was evaluated on the IBM Wikipedia Corpus (~40,000 sentences) and a Persuasive Essays Dataset.

  • IBM Corpus: The Tree Kernel (TK) approach achieved an AUROC of 0.808, significantly outperforming the Bag-of-Words baseline (0.771). When a simple topic-similarity feature was added, the system achieved an $F_1$ score of 18.0, surpassing the original results reported by the IBM team (16.0) on a comparable dataset.
  • Persuasive Essays: The model reached a 71.4 $F_1$ score, demonstrating its robustness in domains with consistent argumentative styles.
  • Domain Generalization: Qualitative tests showed the model could distinguish between controversial and non-controversial Wikipedia articles, successfully identifying the "language of contention" without being told what the controversy was about.

Conclusion and Future Work

By focusing on the structural and rhetorical signatures of claims, this method provides a powerful tool for context-independent argument mining. While challenges remain—particularly the extreme data imbalance in large corpora—the success of tree kernels suggests that the "shape" of an argument is just as important as its content. Future research will look to extend this structural approach to evidence detection and the extraction of relations between different parts of an argument.

Summary updated

BibTeX
@inproceedings{lippi2015context,
  author = {Marco Lippi and Paolo Torroni},
  title = {{Context-Independent Claim Detection for Argument Mining}},
  booktitle = {International Joint Conference on Artificial Intelligence},
  year = {2015},
}
Figure