Proceedings of the Tenth Workshop on Computational Approaches to Subjectivity, Sentiment and Social Media Analysis

Alexandra Balahur, Roman Klinger, Veronique Hoste, Carlo Strapparava, Orphee De ClercqNAACL 2019

TL;DR

This document comprises the proceedings of the Tenth Workshop on Computational Approaches to Subjectivity, Sentiment and Social Media Analysis (WASSA 2019). It showcases research on understanding and analyzing subjective content, opinions, emotions, and social dynamics within text, particularly from social media. The workshop highlights advancements in areas like stance detection, sentiment analysis, humor detection, and the challenges of processing code-mixed languages and resource-lean languages.

What Problem Does This Paper Address?

The collection of papers addresses the growing need to computationally understand subjective and emotional content in text, a challenge amplified by the rise of social media. Traditional NLP tasks often focus on factual information, but a significant portion of online communication involves opinions, sentiments, emotions, and subjective stances. This workshop tackles the complexities of:

  • Subjectivity and Sentiment Analysis (SSA): Moving beyond simple positive/negative classification to more nuanced understanding of opinions, emotions, and their targets.
  • Social Media Analysis: Dealing with the unique characteristics of social media data, such as informality, brevity, code-mixing, and the influence of social interactions.
  • Interdisciplinary Challenges: Bridging the gap between computational linguistics and fields like psychology, sociology, and economics to better model human affect and social phenomena.
  • Fine-grained Analysis: Developing models that can predict not just overall sentiment but also specific aspects, targets, and even subtle emotions like empathy and distress.
  • Resource-Lean Languages: Extending SSA capabilities to languages with limited annotated data.

Methods

The papers presented at WASSA 2019 employ a diverse range of methods, reflecting the state-of-the-art in computational linguistics and NLP:

  • Machine Learning and Deep Learning:
    • Multi-Task Learning: Used for tasks like stance detection in code-mixed languages, allowing a single model to learn multiple related tasks simultaneously, improving generalization.
  • Neural Attention Models: Applied to online abuse detection, enabling models to focus on the most relevant parts of the text.
  • Fine-Tuned Embeddings: Explored for modeling intensifiers in emotion analysis, leveraging pre-trained language models and adapting them to specific linguistic phenomena.
  • Hybrid Models: Combining structured representations with data-driven approaches for tasks like negation and sentiment analysis in customer service conversations.
  • Deep Learning Techniques: Specifically applied to humor detection in code-mixed tweets.
  • Data Augmentation and Labeling Strategies:
    • Soft Label Strategy: Proposed for target-level sentiment classification, offering a more flexible approach to annotation than hard labels.
  • Linguistic Feature Engineering:
    • Incorporating features related to morality to enhance the measurement of social effects.
  • Analyzing the role of negations and intensifiers in sentiment and emotion analysis.
  • Cross-lingual Approaches:
    • Developing methods for subjectivity detection in resource-lean languages, often involving transfer learning or adaptation techniques.
  • Psychologically-Informed Modeling:
    • Developing computational models for empathy and distress that align with psychological definitions, using novel annotation methodologies.

Datasets

The papers utilize a variety of datasets, some standard and some specific to the research presented:

  • Social Media Data: Tweets are a primary source for many papers, particularly for humor detection, stance detection in code-mixed languages, and analyzing emotions related to events.
  • Customer Service Conversations: Used for analyzing negation and sentiment in a specific domain.
  • News Stories: Employed in the keynote talk on modeling empathy and distress.
  • Newly Introduced Benchmarks: The keynote talk by Sven Buechel mentions the construction of the "first publicly available gold standard for empathy prediction," which uses a novel annotation methodology to capture empathy assessments by the writer using multi-item scales. This aims to address limitations in existing empathy prediction datasets.

Key Insights and Ideas

  • Code-Mixing is a Major Hurdle: Analyzing text that mixes multiple languages (e.g., Hindi-English) presents significant challenges for sentiment and stance detection, requiring specialized multi-task learning approaches.
  • Nuance in Sentiment is Crucial: Moving beyond simple polarity to understand target-level sentiment, the role of intensifiers, and the impact of negations is essential for accurate analysis.
  • Empathy is Complex: Computational models need to account for the psychological complexity of empathy, distinguishing between different forms like empathic concern and personal distress, and utilizing robust annotation methods.
  • Humor Detection is Challenging, Especially in Code-Mixed Text: Deep learning techniques show promise, but the informal and context-dependent nature of humor, amplified by code-mixing, makes it a difficult task.
  • Social Effects Can Be Quantified: Concepts like "morality" can be captured and measured to understand their influence on social effects, suggesting a link between linguistic analysis and broader societal impacts.
  • Sarcasm Detection Requires Context: Detecting sarcasm, especially when it involves numerical data, necessitates sophisticated methods that go beyond surface-level text analysis.
  • Resource-Lean Languages Need Dedicated Solutions: Developing effective subjectivity detection for languages with limited annotated data requires innovative approaches, likely involving cross-lingual transfer or adaptation.

Key Results

  • The paper "Stance Detection in Code-Mixed Hindi-English Social Media Data using Multi-Task Learning" was awarded "best paper."
  • The keynote talk introduces a new gold standard for empathy prediction, distinguishing between multiple forms of empathy, which is a significant step towards more accurate computational modeling of human emotions.
  • Several papers demonstrate the effectiveness of deep learning and multi-task learning for tasks like humor detection and stance detection in challenging scenarios (e.g., code-mixed tweets).
  • The workshop accepted 12 out of 19 submissions, indicating a competitive selection process with a focus on high-quality research.

Limitations and Future Work

  • Data Scarcity for Specific Tasks: The introduction notes that the number of submissions was "comparably low in contrast to previous years," potentially due to the proximity of WASSA 2018. This suggests a need for continued efforts to foster research and data creation in the field.
  • Complexity of Affective Phenomena: The introduction highlights that "dealing with affective phenomena in text has proven to be a complex, interdisciplinary problem that remains far from being solved." This implies that future work will continue to explore more sophisticated models and interdisciplinary collaborations.
  • Need for Robust Annotation: The keynote talk emphasizes the limitations of existing empathy datasets and the need for novel annotation methodologies that capture nuanced human states. This points to future work in developing better annotation frameworks.
  • Challenges in Code-Mixed and Resource-Lean Languages: The accepted papers indicate ongoing research into these areas, suggesting future work will focus on improving models for multilingual and low-resource settings.
  • Real-World Applications: While many papers mention applications, further research is needed to translate these computational models into robust and deployable systems for various real-world problems.

Summary updated

BibTeX
@inproceedings{balahur2019proceedings,
  author = {Alexandra Balahur and Roman Klinger and Veronique Hoste and Carlo Strapparava and Orphee De Clercq},
  title = {{Proceedings of the Tenth Workshop on Computational Approaches to Subjectivity, Sentiment and Social Media Analysis}},
  booktitle = {Conference of the North American Chapter of the Association for Computational Linguistics},
  year = {2019},
}
Figure