Towards Argument Mining for Social Good: A Survey

Eva Maria Vecchi, Neele Falk, Iman Jundi, Gabriella LapesaACL 2021

TL;DR

This survey argues that Argument Mining (AM) needs to shift its focus from simply identifying persuasive arguments to understanding how arguments contribute to productive, cooperative discourse, especially in social media. They propose a novel, integrated definition of "argument quality" that incorporates "deliberative quality" and suggest "semi-automatic moderation" as a key application for achieving "social good."

What Problem Does This Paper Address?

Traditional Argument Mining (AM) research often focuses on identifying claims and premises and determining their persuasive strength. However, this overlooks the broader goal of constructive dialogue, particularly in the complex and often chaotic environment of social media. The authors argue that AM needs to move beyond individual argument persuasiveness to consider how arguments contribute to a shared goal and foster productive deliberation. This is especially crucial for applications aimed at "social good," where the aim is not to "win" an argument but to facilitate understanding and cooperation. The paper addresses the challenge of defining and measuring "argument quality" in a way that is relevant to these broader social goals, particularly in the context of online discussions and decision-making.

Methods

The paper presents a survey of existing Argument Mining (AM) research, re-framing it through the lens of deliberation and social good. The core methodological contribution is a proposed novel definition of "argument quality" that integrates concepts from both AM and Social Sciences' "deliberative quality." This integrated definition assesses contributions at multiple levels: the contribution itself, its context, and its consequential effect on the discourse. The authors advocate for a shift from a purely persuasive focus to one that emphasizes cooperation and inclusivity. They also propose "semi-automatic moderation" as a practical application that serves as a testbed for their integrated quality definition, leveraging existing NLP techniques like hate speech detection and fact-checking.

Datasets

The paper discusses various datasets and corpora used in AM research, including:

  • Twitter corpora for argument component extraction.
  • Datasets for argument extraction and classification in German, English, and Chinese.
  • Corpora for argument quality ranking.
  • Datasets for detecting fallacies and implicit warrants.
  • Datasets for hate speech detection, fact-checking, and framing detection.

The paper does not introduce a new dataset but rather surveys existing work that utilizes various datasets to address different aspects of argument mining.

Key Insights and Ideas

  • Argument Mining for Social Good requires a paradigm shift: The goal isn't to find the "perfect debater" but the "best team player" who contributes to a shared goal.
  • Persuasiveness isn't enough: Argument quality needs to go beyond how convincing an argument is to consider its contribution to productive discourse and inclusivity.
  • Integrating Deliberative Quality is key: Social science concepts of deliberation, focusing on respect, equality, participation, and interactivity, are crucial for a holistic understanding of argument quality in social contexts.
  • Social media presents unique challenges and opportunities: The chaotic nature of social media data requires robust AM methods, but its vastness offers significant potential for social impact.
  • Semi-automatic moderation is a prime application: This task naturally tests the integrated definition of argument quality and has direct societal benefits.

Key Results

The paper is a survey and conceptual proposal, not an empirical study with quantitative results. Its key "results" are conceptual:

  • A proposed integrated definition of argument quality that combines textual properties, contextual relations, and deliberative aspects.
  • An argument for the necessity of considering the "cooperation challenge" in argument quality assessment.
  • The identification of semi-automatic moderation as a promising application for AM in social good, highlighting its potential to quantify argument/deliberative quality and its societal benefits.

Limitations and Future Work

  • Ambition of Semi-automatic Moderation: The authors acknowledge that semi-automatic moderation is "extremely ambitious" and presents significant practical challenges.
  • Bias in NLP: The paper touches upon the risk of bias in NLP models, particularly in how they might infer negative impacts on argument quality from cues related to user groups or literacy levels, which needs careful consideration.
  • Representing Discourse: A key open question identified is how to effectively represent argumentative discourse, both within individual contributions and across entire discussions, suggesting this as a future research direction.
  • Scaling AM: The paper highlights the ongoing challenge of scaling AM to handle the vastness and heterogeneity of social media data.
  • Multilingual AM: While progress is being made, there's a need for more datasets and analysis in non-Indo-European languages to ensure inclusivity.

Summary updated

BibTeX
@inproceedings{vecchi2021towards,
  author = {Eva Maria Vecchi and Neele Falk and Iman Jundi and Gabriella Lapesa},
  title = {{Towards Argument Mining for Social Good: A Survey}},
  booktitle = {Annual Meeting of the Association for Computational Linguistics},
  year = {2021},
}
Figure