A Multi-Task Learning Approach to Sarcasm Detection (Student Abstract)

Edoardo Savini, Cornelia CarageaAAAI 2020 · DOI

Introduction

Sarcasm represents a significant challenge for Natural Language Processing (NLP) because it relies on subtle linguistic incongruity where the intended meaning is the opposite of the literal words. For sentiment analysis systems, sarcasm often acts as a "noise" factor; a sentence like "I love waiting in line for three hours" appears positive to basic algorithms but is clearly negative to a human observer.

Existing approaches typically rely on extensive feature engineering or complex contextual modeling, such as "user embeddings" that profile a user's past behavior. This research explores whether sarcasm detection can be improved using only the text of an utterance by leveraging its inherent relationship with sentiment. Since sarcastic statements are almost always associated with implied negative sentiment, the authors propose a Multi-Task Learning (MTL) framework where a model trained to understand sentiment naturally develops better internal features for spotting sarcasm.

Methodology

The proposed framework utilizes Multi-Task Learning to train a model on two tasks simultaneously, sharing a portion of the network to learn generalized features that benefit both objectives.

The Multi-Task Learning architecture featuring a shared BiLSTM backbone and dual MLP heads for sarcasm and sentiment tasks.

Model Architecture

  • The Backbone: The model employs a shared embedding layer followed by a Bidirectional Long Short Term Memory (BiLSTM) network. This architecture is designed to capture sequential dependencies and context within a sentence.
  • Embeddings: To capture both fixed meanings and context-dependent nuances, the model concatenates ELMo (contextual) and FastText (non-contextual) word embeddings.
  • Task Heads: Following the BiLSTM, the model splits into two separate Multi-Layer Perceptrons (MLPs):
  • Primary Task: Sarcasm detection (Binary classification: Sarcastic vs. Not Sarcastic).
  • Auxiliary Task: Sentiment classification (A 5-point scale from Very Negative to Very Positive).
  • Loss Function: The total loss is a weighted sum of the cross-entropy losses from both tasks, controlled by a hyperparameter ($\lambda$) that determines the influence of the sentiment task during training.

Experimental Setup

The study utilizes the SARC (Self-Annotated Reddit Corpus), a massive dataset of Reddit comments. Sarcasm labels are "gold labels" provided by users (via the /s tag). Because SARC lacks sentiment tags, the authors generated "silver labels" for the auxiliary task using a pre-trained Stanford NLP sentiment model. Evaluation was conducted on the "Main Balanced" variant of SARC to ensure an equal distribution of sarcastic and non-sarcastic examples.

Results and Discussion

The MTL approach demonstrated clear quantitative gains over single-task baselines and traditional models.

Comparison of F1-scores across various models, highlighting the performance of the proposed BiLSTM + MTL approach.

As shown in the results, the BiLSTM + MTL model achieved an F1-score of 0.763, significantly outperforming the vanilla BiLSTM (0.748) and standard text-only baselines like Bag-of-words (0.64) and CNNs (0.66). Most notably, the proposed model reached within 0.7% of the state-of-the-art CASCADE model (0.77). While CASCADE requires complex user personality features and history, the MTL model achieves comparable performance using zero information about the user, relying solely on the text of the message.

Conclusion

This research demonstrates that sentiment is a functional component of sarcasm. By forcing the model to predict sentiment as an auxiliary task, it learns to identify emotional weight—a prerequisite for recognizing when that weight is being used ironically. While the model currently relies on "silver labels" for sentiment, future work could involve explicit modeling of the contradiction between literal sentiment and intended sarcasm to further close the gap with context-heavy models.

Summary updated

BibTeX
@inproceedings{savini2020multi,
  author = {Edoardo Savini and Cornelia Caragea},
  title = {{A Multi-Task Learning Approach to Sarcasm Detection (Student Abstract)}},
  booktitle = {AAAI Conference on Artificial Intelligence},
  year = {2020},
  doi = {10.1609/aaai.v34i10.7226},
}
Figure