Abstract
Understanding the pragmatic flow of online discussions is a significant challenge due to the informal and asynchronous nature of social media. This paper introduces a novel hierarchical LSTM model equipped with a "word relevance" attention mechanism designed to classify discourse acts—such as questions, answers, or disagreements—within conversation threads. Unlike previous approaches that rely on platform-specific structural metadata, this model focuses purely on textual content, achieving state-of-the-art domain independence across diverse platforms like Reddit and Facebook.
Introduction
Social media conversations are complex, often involving non-linear structures and informal language. While prior research has targeted specific tasks like sarcasm detection, this work addresses the broader challenge of complete discourse act classification. The goal is to categorize every comment in a thread based on its functional role (e.g., Elaboration, Negative Reaction, Humor).
A primary hurdle in this field is "domain dependence." Many models utilize structural features unique to specific platforms, such as Reddit's tree depth or upvote counts, which limits their generalizability. The authors propose a model that understands the underlying language of discourse, allowing it to function effectively across different social media environments without relying on platform-specific structural features.
Methodology
The proposed architecture is a 2-Dimensional Hierarchical LSTM featuring a specialized attention mechanism:
- Word-Dimension LSTM: This layer processes the sequence of words within an individual comment to generate a vector representation.
- Comment-Dimension LSTM: This layer processes the sequence of comment vectors in a conversation chain, capturing how the role of a comment depends on its predecessors.
- Word Relevance Attention: This core novelty allows the model to "peek" at the thread-starting comment (the topic) and the parent comment (the immediate context). By using TF-IDF-weighted vectors from these sources, the model identifies which words in the current comment are most relevant to the ongoing discourse.
Experimental Setup and Datasets
The model was evaluated on two primary datasets:
- Coarse Discourse Dataset (Reddit): A large-scale dataset with over 100,000 comments labeled across nine discourse acts, including Announcement, Question, Answer, and Disagreement.
- Facebook Dataset: A set of 20 threads (1,177 comments) covering controversial topics like Brexit, manually annotated to test the model's cross-platform generalization.
Results and Key Insights
The 2D LSTM with attention significantly outperformed baselines, achieving an F1-score of 0.71 on Reddit and 0.66 on Facebook. Notably, it surpassed previous state-of-the-art models that relied on platform-specific structural features.
Key findings include:
- LSTMs vs. CNNs: While CNNs excel at identifying local patterns (e.g., "Why..." for questions), LSTMs are superior for capturing long-distance dependencies required for argumentative roles like Disagreement.
- Unsupervised Connective Discovery: The attention mechanism naturally learned to weight "discourse connectives" like "Why" or "Serious?" based on context, acting as an unsupervised detector.
- Subjectivity: Human experiments highlighted the inherent ambiguity of discourse; annotators often found the model's "incorrect" predictions to be plausible interpretations.
Conclusion
This research demonstrates that a text-centric hierarchical approach can effectively model the flow of social media discussions across different domains. Future work will explore processing full tree structures rather than linear chains and incorporating speaker profiles to better resolve ambiguities in roles like humor or sarcasm.
Summary updated