How Did the Discussion Go: Discourse Act Classification in Social Media Conversations

Subhabrata Dutta, Tanmoy Chakraborty, Dipankar DasUnsupervised 2018 · DOI · arXiv · OpenAlex

Abstract

Understanding the pragmatic flow of online discussions is a significant challenge due to the informal and asynchronous nature of social media. This paper introduces a novel hierarchical LSTM model equipped with a "word relevance" attention mechanism designed to classify discourse acts—such as questions, answers, or disagreements—within conversation threads. Unlike previous approaches that rely on platform-specific structural metadata, this model focuses purely on textual content, achieving state-of-the-art domain independence across diverse platforms like Reddit and Facebook.

Introduction

Social media conversations are complex, often involving non-linear structures and informal language. While prior research has targeted specific tasks like sarcasm detection, this work addresses the broader challenge of complete discourse act classification. The goal is to categorize every comment in a thread based on its functional role (e.g., Elaboration, Negative Reaction, Humor).

Examples of discourse acts in a social media conversation thread.

A primary hurdle in this field is "domain dependence." Many models utilize structural features unique to specific platforms, such as Reddit's tree depth or upvote counts, which limits their generalizability. The authors propose a model that understands the underlying language of discourse, allowing it to function effectively across different social media environments without relying on platform-specific structural features.

Methodology

The proposed architecture is a 2-Dimensional Hierarchical LSTM featuring a specialized attention mechanism:

  1. Word-Dimension LSTM: This layer processes the sequence of words within an individual comment to generate a vector representation.
  2. Comment-Dimension LSTM: This layer processes the sequence of comment vectors in a conversation chain, capturing how the role of a comment depends on its predecessors.
  3. Word Relevance Attention: This core novelty allows the model to "peek" at the thread-starting comment (the topic) and the parent comment (the immediate context). By using TF-IDF-weighted vectors from these sources, the model identifies which words in the current comment are most relevant to the ongoing discourse.
Architecture of the hierarchical LSTM for discourse act classification.

Experimental Setup and Datasets

The model was evaluated on two primary datasets:

  • Coarse Discourse Dataset (Reddit): A large-scale dataset with over 100,000 comments labeled across nine discourse acts, including Announcement, Question, Answer, and Disagreement.
  • Facebook Dataset: A set of 20 threads (1,177 comments) covering controversial topics like Brexit, manually annotated to test the model's cross-platform generalization.

Results and Key Insights

The 2D LSTM with attention significantly outperformed baselines, achieving an F1-score of 0.71 on Reddit and 0.66 on Facebook. Notably, it surpassed previous state-of-the-art models that relied on platform-specific structural features.

Key findings include:

  • LSTMs vs. CNNs: While CNNs excel at identifying local patterns (e.g., "Why..." for questions), LSTMs are superior for capturing long-distance dependencies required for argumentative roles like Disagreement.
  • Unsupervised Connective Discovery: The attention mechanism naturally learned to weight "discourse connectives" like "Why" or "Serious?" based on context, acting as an unsupervised detector.
  • Subjectivity: Human experiments highlighted the inherent ambiguity of discourse; annotators often found the model's "incorrect" predictions to be plausible interpretations.

Conclusion

This research demonstrates that a text-centric hierarchical approach can effectively model the flow of social media discussions across different domains. Future work will explore processing full tree structures rather than linear chains and incorporating speaker profiles to better resolve ambiguities in roles like humor or sarcasm.

Summary updated

BibTeX
@inproceedings{dutta2018how,
  author = {Subhabrata Dutta and Tanmoy Chakraborty and Dipankar Das},
  title = {{How Did the Discussion Go: Discourse Act Classification in Social Media Conversations}},
  booktitle = {Unsupervised},
  year = {2018},
  doi = {10.1007/978-3-030-01872-6\_6},
}
Figure