Advances in Debating Technologies: Building AI That Can Debate Humans

Roy Bar-Haim, Liat Ein-Dor, Matan Orbach, Elad Venezian, Noam SlonimACL 2021

TL;DR

This tutorial paper introduces the cutting-edge field of computational argumentation, specifically focusing on building AI systems capable of engaging in live debates with humans. It outlines the scientific challenges and a comprehensive pipeline for such systems, drawing heavily on IBM's Project Debater as a case study and offering practical tools for researchers.

What Problem Does This Paper Address?

The paper addresses the significant challenge of creating Artificial Intelligence systems that can not only understand but also actively participate in human-like debates. While AI has made strides in various cognitive tasks, engaging in a dynamic, persuasive, and responsive debate requires a complex interplay of skills, including argument retrieval, quality assessment, stance detection, narrative construction, and real-time rebuttal. The authors highlight that existing research often tackles these problems in isolation, lacking a holistic view of the entire debating system. They aim to bridge this gap by presenting a structured approach and the underlying scientific problems that need to be solved to build a truly debating AI.

Methods

The tutorial outlines a comprehensive pipeline for a debating system, which can be broken down into several key stages:

  • Argument Mining: This involves identifying relevant arguments and their components (claims, premises) from large corpora or specific documents related to a debate topic.
  • Argument Evaluation and Analysis: This includes assessing the quality of arguments and classifying their stance (supporting or opposing a given side).
  • Modeling Human Dilemma: This addresses the use of "principled arguments" – common, abstract arguments that can be applied across various topics (e.g., the "black market" argument). The paper discusses creating taxonomies of these arguments and matching them to debate contexts.
  • Listening Comprehension and Rebuttal: A crucial and challenging aspect, this involves real-time understanding of spoken arguments, identifying key points, and generating persuasive counter-arguments. This is contrasted with simpler conversational agents.
  • Core NLP Capabilities: The paper mentions underlying NLP capabilities like thematic clustering, Wikification, and semantic similarity, which are essential for various stages of the pipeline.
  • From Arguments to Narrative: This stage focuses on organizing mined and generated arguments into a coherent, persuasive narrative that can hold an audience's attention. It also touches upon the role of humor.
  • Applications and Implications: The paper discusses practical applications like "Speech by Crowd" for decision support and "Key Point Analysis" for summarizing large argument collections.

Datasets

The paper mentions that IBM's Project Debater has resulted in "many datasets freely available for research purposes." However, specific details about these datasets, their size, or their nature are not elaborated upon in this tutorial abstract. No new datasets are introduced.

Key Insights and Ideas

  • Debating AI is a Grand Challenge: Building an AI that can debate humans is presented as a significant undertaking, comparable to previous AI milestones like Deep Blue and Watson.
  • Holistic System Design is Crucial: The authors emphasize the need to view debating systems as a complete pipeline, rather than focusing on isolated components, to understand the information flow and interactions.
  • Real-time Rebuttal is a Major Hurdle: The ability to understand and respond to spoken arguments in real-time, especially when arguments are implicit or span multiple sentences, is identified as a particularly difficult challenge.
  • Principled Arguments Add Depth: Incorporating abstract, principled arguments that can be applied across diverse topics is key to creating more robust and human-like debating strategies.
  • Practical Tools are Emerging: The availability of Project Debater components as cloud APIs signifies a move towards making these advanced capabilities accessible for academic and practical use.

Key Results

Not discussed in the paper. This is a tutorial abstract outlining the scope and content of a presentation, not a research paper reporting specific experimental results.

Limitations and Future Work

Limitations: * The paper acknowledges that the academic literature often addresses debating problems in isolation, lacking a holistic view. * The challenge of real-time listening comprehension and rebuttal for spoken arguments is highlighted as a significant difficulty.

Future Work: * The tutorial aims to discuss practical applications and future challenges of debating technologies. * Specific future directions mentioned include "Speech by Crowd" and "Key Point Analysis." * The availability of Project Debater APIs suggests a future where researchers can readily build upon these advancements.

Summary updated

BibTeX
@inproceedings{barhaim2021advances,
  author = {Roy Bar-Haim and Liat Ein-Dor and Matan Orbach and Elad Venezian and Noam Slonim},
  title = {{Advances in Debating Technologies: Building AI That Can Debate Humans}},
  booktitle = {Annual Meeting of the Association for Computational Linguistics},
  year = {2021},
}
Figure