Structured Representation Learning for Online Debate Stance Prediction

Chang Li, Aldo Porco, Dan GoldwasserCOLING 2018

TL;DR

This paper introduces a novel approach to predicting stances in online debates by framing it as a structured representation learning problem. By jointly embedding debate participants, their posts, and stance labels into a shared space, the model captures complex relationships and achieves state-of-the-art performance, outperforming previous collective classification methods.

What Problem Does This Paper Address?

Online debates are a rich source of information about diverse perspectives on various issues. However, automatically understanding the stances expressed within these debates is a significant challenge. This difficulty arises from the need to model not only the textual content of individual posts but also the intricate conversational interactions between users. Existing methods often treat posts in isolation or use collective classification approaches that can overlook the nuanced relationships between different debate topics. This paper aims to bridge this gap by developing a more holistic method for stance prediction that leverages both textual and structural information within debates.

Methods

The core of this paper's contribution lies in its structured representation learning approach. Instead of relying on traditional feature engineering or isolated classification, the authors propose to learn a unified embedding space for all relevant entities: authors, textual posts, and stance labels (e.g., "Pro" or "Con").

Key methodological aspects include:

  • Joint Representation Learning: The model learns to embed authors, posts, and stance labels into the same vector space. This allows for direct comparison of their representations, enabling stance prediction by measuring the similarity between a post's embedding and the embeddings of potential stance labels.
  • Capturing Structural Dependencies: The learning process is designed to respect the inherent relationships within debates. This includes modeling the connection between an author and their posts, as well as the disagreements or agreements between consecutive posts.
  • Global MAP Inference: For prediction, the model employs a collective classification approach using a factor graph. This allows for joint decision-making across all posts in a debate thread, ensuring consistency and leveraging the learned representations.
  • Global Embedding Learning: To further enhance performance, the authors integrate the inference process directly into the training loop. This "global training" approach uses a structured hinge loss to promote consistency between predicted stances and gold labels, ensuring that the learned representations are optimized for the overall prediction task.
  • Integer Linear Programming (ILP) Formulation: For prediction, an ILP formulation is used to enforce constraints and preferences, such as ensuring an author maintains a consistent stance across their posts.

Datasets

The proposed model was evaluated on two datasets from the Internet Argumentation Corpus:

  • 4FORUMS: This dataset comprises debates from the 4forums.com website.
  • CREATEDEBATE: This dataset is from the createdebate.com website.

These datasets cover eight distinct topics, including Abortion, Gay Rights, Marijuana, Obama, Evolution, Gay Marriage, and Gun Control. The paper notes that for CREATEDEBATE, they used the same data splits as a previous study (Hasan and Ng, 2013) for comparability, while for 4FORUMS, they randomly split the data.

Key Insights and Ideas

  • Unified Embedding Space is Powerful: The central idea that embedding authors, posts, and stances into a single semantic space is highly effective for capturing complex relationships and improving prediction accuracy.
  • Leveraging "Cooperative" Information: The model demonstrates that information from different views (e.g., author-text, text-text) can be made "cooperative" during training, leading to more robust and generalizable representations.
  • Global Learning Outperforms Inference: Integrating the inference process into the training (global learning) significantly improves performance over simply performing inference on a jointly embedded representation. This suggests that optimizing for the global prediction task directly is more beneficial.
  • Constraints Enhance Performance: The use of constraints, such as author consistency and disagreement between consecutive posts, leads to substantial performance gains, highlighting the importance of incorporating domain-specific knowledge.
  • Cross-Topic Generalization: The model shows promise in generalizing across different debate topics, especially when trained with additional information, suggesting that underlying ideological connections can be captured.

Key Results

The paper reports significant improvements in stance prediction accuracy across both datasets.

  • Outperforming Baselines: The proposed "Global" model consistently outperforms previous state-of-the-art models, including collective classification approaches like NB and CRF (Hasan and Ng, 2013) and PSL (Sridhar et al., 2015), on most topics.
  • CREATDEBATE: For instance, on the CREATEDEBATE dataset, the Global model achieves accuracies in the mid-70s to low-80s across various topics, often surpassing the best reported results.
  • 4FORUMS: Similarly, on the 4FORUMS dataset, the Global model achieves accuracies in the high-80s, significantly outperforming other methods.
  • Impact of Constraints: The study shows that incorporating author constraints (AC) and consecutive constraints leads to notable performance increases, with "Inference(Both)" generally outperforming "Inference(AC)" and "Inference(Consecutive)" individually.
  • User Attributes and Cross-Domain: Augmenting the model with user attributes and training on both in-domain and out-of-domain data further boosts performance, demonstrating the model's ability to leverage additional information for better generalization.

Limitations and Future Work

  • Computational Cost: The paper acknowledges that computing gradients for certain loss terms (e.g., author vs. text) can be computationally expensive due to the large number of authors and texts. They address this using negative sampling.
  • Supervision: The authors note that obtaining sufficient supervision is a bottleneck in NLP. They suggest future work could explore applying their approach in weakly and distantly supervised settings to alleviate this difficulty.
  • Additional Aspects: The model's flexibility allows for the inclusion of additional views or aspects beyond author, text, and stance. Future work could explore incorporating other relevant information to further improve generalization.
  • Broader Applicability: The authors believe their structured representation learning approach is applicable to other domains with complex inter-connected structures, such as other conversation analysis tasks, shared representations of text and images, and information networks like citation graphs and social networks.

Summary updated

BibTeX
@inproceedings{li2018structured,
  author = {Chang Li and Aldo Porco and Dan Goldwasser},
  title = {{Structured Representation Learning for Online Debate Stance Prediction}},
  booktitle = {International Conference on Computational Linguistics},
  year = {2018},
}

Loading related content...

Figure