HARGAN: Heterogeneous Argument Attention Network for Persuasiveness Prediction

Kuo Yu Huang, Hen-Hsen Huang, Hsin-Hsi ChenAAAI 2021 · DOI

Introduction

Predicting whether an argument will change a reader's mind is a complex task in Natural Language Processing. Unlike simple sentiment analysis, persuasiveness in multi-turn debates depends on the logical interplay between claims and premises. This paper, presented at AAAI-21, introduces HARGAN, a framework that explicitly models these argumentative structures to predict persuasion outcomes in online forums like Reddit’s r/ChangeMyView.

The Challenge of Argumentative Structure

In multi-turn debates, persuasion is a process rather than a single event. Existing models often treat text as a flat sequence or focus on local interactions, missing the broader "argument structure"—the web of elementary discourse units (ADUs) that support or attack one another. Capturing these long-range, structural dependencies is crucial for understanding the dynamics of a successful argument.

The HARGAN Framework

The authors propose a two-stage pipeline to bridge the gap between raw text and persuasive impact.

Stage 1: Argument Structure Parsing

The first stage identifies the building blocks of the debate:

  1. ADU Recognition: A BERT-based model with a Conditional Random Field (CRF) layer identifies spans of text acting as claims or premises using a "BIEO" (Beginning, Inside, Ending, Outside) tagging scheme.
  2. Dependency Parsing: The system predicts relationships between these units, labeling them as "Support" or "Attack."
The two-stage pipeline of HARGAN, illustrating the transition from raw text to argument structure parsing and final persuasiveness prediction.

Stage 2: Heterogeneous Argument Attention Network

The core of the model is a heterogeneous graph where nodes represent the Topic, Paragraphs, and individual ADUs.

  • Relational Edges: HARGAN uses diverse edge types, including Support, Attack, Non-polar (uncertain links), and Hierarchical (connecting ADUs to paragraphs). This allows information to flow even when the parser is uncertain.
  • Attention Mechanism: A Relational Heterogeneous Graph Attention Network (HAN) aggregates information, allowing the model to weigh specific argumentative relations more heavily based on their relevance to persuasion.
  • Multi-task Learning: The model jointly learns to predict persuasiveness and the stance (viewpoint) of each participant. This auxiliary task helps the model identify the "sides" of a debate, achieving approximately 97% accuracy in stance prediction.
Detailed architecture of the HARGAN model, illustrating the graph aggregation module and the multi-task learning components for persuasiveness and stance prediction.

Experimental Evaluation

The model was evaluated on the ChangeMyView (CMV) dataset using a pairwise comparison task: given two conversation threads under the same original post, can the model identify which one successfully convinced the original poster? The processed dataset included approximately 15,000 training pairs and 5,000 test pairs.

Results and Key Insights

HARGAN achieved a state-of-the-art accuracy of 85.14%, a significant improvement over the previous leading model, DTDMN, which achieved 76.38%. Key findings include:

  • Structure is Paramount: Explicitly modeling the logic (the tree of claims) provides a much stronger signal than text encoding alone. Ablation studies showed that removing the graph aggregation module caused accuracy to drop by over 15%.
  • Stance as a Catalyst: Forcing the model to identify speaker stance significantly improves its grasp of attack/support dynamics.
  • Robustness: The inclusion of "non-polar" links helps mitigate error propagation from the initial parsing stage, ensuring the graph remains informative even with imperfect parsing.

Conclusion

HARGAN demonstrates that the key to predicting persuasiveness lies in the structural relationships between arguments. By transforming debates into heterogeneous graphs and applying relational attention, the model captures the nuances of human logic more effectively than previous sequence-based approaches. Future work may involve applying this framework to recommendation systems or automated debate assistants.

Summary updated

BibTeX
@inproceedings{huang2021hargan,
  author = {Kuo Yu Huang and Hen-Hsen Huang and Hsin-Hsi Chen},
  title = {{HARGAN: Heterogeneous Argument Attention Network for Persuasiveness Prediction}},
  booktitle = {AAAI Conference on Artificial Intelligence},
  year = {2021},
  doi = {10.1609/aaai.v35i14.17542},
}
Figure