Summarizing Multi-Party Argumentative Conversations in Reader Comment on News

Emma Barker, Robert J. Gaizauskas

TL;DR

This paper introduces a novel framework for analyzing and graphically representing the argumentative structure of online news reader comments. It then proposes methods to generate summaries that capture the essence of these complex, multi-party debates, moving beyond simple topic-based extraction to reflect the underlying arguments and viewpoints.

What Problem Does This Paper Address?

Online news articles often feature comment sections that quickly grow into massive, multi-party conversations. These discussions are frequently argumentative, with readers questioning, supporting, or refuting points made by others or the original article. However, the sheer volume of comments makes it nearly impossible for a reader to grasp the core arguments and viewpoints. Existing summarization methods for these comment sections are typically extractive and topic-based, failing to capture the argumentative nature of the discussions. They don't identify the key issues being debated, the different stances taken, or the evidence presented, leading to summaries that lack depth and argumentative insight.

Methods

The authors propose a two-pronged approach:

  1. Argument Analysis Framework: They introduce a "light-weight analytical framework" to characterize arguments in reader comments. This framework identifies three key elements:
  2. Issues: Questions or problems with multiple contending answers (viewpoints). These are often implicit and emerge from the discussion.
  3. Viewpoints: Assertions that contradict or contend with another assertion (either from another comment or the article).
  4. Assertions: Propositions put forward by a commenter, which can serve as viewpoints or provide rationales. They also define relations between these elements, such as "rationale" (providing evidence) and "agree/disagree."

  5. Graphical Representation: To visualize the argumentative structure, they propose a directed graph. Nodes represent issues, viewpoints, or assertions, with edges indicating their relationships. Implicit elements are also represented. This graphical representation grounds the analysis in the textual content by linking nodes to specific sentences or comments.

  6. Summarization Algorithms: Based on the argument graph, they outline methods for generating argumentative summaries. They identify quantitative and structural features of the graph (e.g., issue depth, viewpoint support count) that can inform summary generation. They present two heuristic algorithms:
  7. Simple Issue-oriented Summarizer: Selects issues based on quantitative measures and then lists viewpoints and rationales.
  8. Simple Argument-Oriented Summarizer (Algorithm 1): A more sophisticated approach that iteratively selects issues, viewpoints, and supporting rationales based on defined ordering and thresholding functions, aiming to capture the argumentative flow.

Datasets

The paper uses an example from The Guardian newspaper regarding a town council's decision to reduce bin collection frequency. They also refer to an extended version of this example for tables and figures, which includes the original article and the first 30 comments. They mention the "SENSEI corpus of human summaries of reader comment conversations in on-line news" as a source for gold-standard human summaries, which they use for comparison.

Key Insights and Ideas

  • The centrality of "Issues": The authors emphasize that issues, often implicit, are the core around which arguments revolve. Their framework places issues at the forefront, unlike many other argument mining approaches.
  • Argumentative structure is key for summarization: Simply extracting popular or frequent comments doesn't convey the argumentative essence. A summary needs to reflect why people are arguing and what they are arguing about.
  • Graphical representation as a bridge: The proposed argument graph acts as an intermediate representation, translating the complex, often implicit, argumentative structure of comments into a more manageable and analyzable form.
  • Summarization as a downstream task: Argument mining, traditionally focused on identification and classification, can be effectively leveraged for practical applications like summarization, providing a more informative overview of online discussions.

Key Results

The paper presents a framework and algorithms, rather than empirical results with quantitative metrics. They demonstrate their framework by constructing an argument graph for a specific news article and its comments (Figure 2). They also provide examples of human-authored summaries and a potential automatic summary (Figure 3), suggesting that their approach can generate summaries that align with human-generated ones, albeit with modest direction. The comparison with human summaries serves as a qualitative validation of their approach.

Limitations and Future Work

The authors acknowledge that their framework, graphical representation, and summarization algorithms are theoretical preliminaries. They need to be implemented and empirically evaluated. Key future work includes:

  • Reliable argument graph extraction: Developing methods to automatically extract argument graphs from news articles and comments, which they break down into sub-tasks like candidate assertion detection, argumentative relation detection, and issue identification.
  • Implementation and evaluation of summarization algorithms: Building and testing the proposed summarization algorithms.
  • Refining the analytical framework: Further validation through multiple analysts generating argument graphs for a corpus of comment sets.
  • Handling disconnected graphs: Extending algorithms to summarize comment sets that might give rise to multiple, unrelated root issues.

Summary updated

Abstract

Existing approaches to summarizing multi-party argumentative conversations in reader comment are extractive and fail to capture the argumentative nature of these conversations. Work on argument mining proposes schemes for identifying argument elements and relations in text but has not yet addressed how summaries might be generated from a global analysis of a conversation based on these schemes. In this paper we: (1) propose an issue-centred scheme for analysing and graphically representing argument in reader comment discussion in on-line news, and (2) show how summaries capturing the argumentative nature of reader comment can be generated from our graphical representation.
BibTeX
@inproceedings{barker2016summarizing,
  author = {Emma Barker and Robert J. Gaizauskas},
  title = {{Summarizing Multi-Party Argumentative Conversations in Reader Comment on News}},
  booktitle = {ArgMining @ ACL},
  year = {2016},
  doi = {10.18653/v1/w16-2802},
}
Figure