Perceived and Intended Sarcasm Detection with Graph Attention Networks

Joan Plepi, Lucie Flek

Abstract

Existing sarcasm detection systems focus on exploiting linguistic markers, context, or user-level priors. However, social studies suggest that the relationship between the author and the audience can be equally relevant for the sarcasm usage and interpretation. In this work, we propose a framework jointly leveraging (1) a user context from their historical tweets together with (2) the social information from a user’s neighborhood in an interaction graph, to contextualize the interpretation of the post. We distinguish between perceived and self-reported sarcasm identification. We use graph attention networks (GAT) over users and tweets in a conversation thread, combined with various dense user history representations. Apart from achieving state-of-the-art results on the recently published dataset of 19k Twitter users with 30K labeled tweets, adding 10M unlabeled tweets as context, our experiments indicate that the graph network contributes to interpreting the sarcastic intentions of the author more than to predicting the sarcasm perception by others.

Introduction

Sarcasm detection is notoriously difficult for artificial intelligence because it relies heavily on social context that text alone cannot capture. Most existing systems treat tweets as isolated snippets, searching for linguistic markers or sentiment shifts. However, sarcasm is inherently social; its meaning depends on the author's history and their relationship with the audience. This research addresses two major gaps: contextual blindness in existing models and the distinction between intended sarcasm (the author's intent) and perceived sarcasm (the audience's interpretation).

Methodology

The proposed framework leverages a heterogeneous graph-based approach to capture both social and historical context. The architecture integrates multiple data streams to contextualize individual posts:

  1. Embeddings: SentenceBERT is used for initial tweet representations, while user2vec encodes user "personalities" based on a history of 10 million unlabeled tweets. This captures a user's typical style and behavioral traits.
  2. Social Graph: A graph is constructed with Users and Tweets as nodes. Edges represent social interactions (mentions, quotes, replies), conversation threads, and authorship relations.
  3. Graph Attention Networks (GAT): The core engine uses an attention mechanism to assign weights to different neighbors. This allows the model to prioritize conversational context when a user's historical data is insufficient or inconsistent.
Overview of the proposed GAT-based architecture for sarcasm detection.

Experimental Setup

The study utilizes the SPIRS dataset, containing roughly 30,000 labeled tweets. A unique feature of SPIRS is the inclusion of "cue tweets" (replies that explicitly identify sarcasm) and "oblivious tweets" (where the audience missed the sarcasm). The authors expanded this dataset by fetching the timelines of 57,000 users, totaling 10 million historical tweets to build robust user profiles.

Results

The GAT-based model demonstrated a significant performance leap over standard benchmarks. As shown in the results table, the proposed model achieved an F1-score of 84.2%, a substantial improvement over the 69.9% achieved by a fine-tuned BERT model.

Performance comparison of various models on the sarcasm detection task.

Notably, the GAT model achieved these results with only 500,000 trainable parameters, compared to BERT's 110 million. This efficiency underscores the value of high-quality social context over raw model size. Ablation studies confirmed that while adding user history (BERT + user2vec) improved performance to 73.4%, the full graph structure was necessary to exceed the 80% F1-score threshold.

Discussion and Key Insights

  • Sarcasm Homophily: The findings suggest that sarcasm is "contagious"; sarcastic users tend to interact with one another. Modeling these communities allows the GAT to identify sarcastic tendencies even in ambiguous text.
  • Intent vs. Perception: One of the most striking findings is that models (and humans) are much better at identifying when an author intended to be sarcastic than predicting if an audience will perceive it correctly. Perceived sarcasm is more subjective and depends on the reader's own biases.
  • Context as a Safety Net: Attention weights showed that when a user's historical data was insufficient, the model automatically shifted its focus to the conversational thread (the social neighborhood) to make a determination.

Conclusion

This work demonstrates that social context is a critical component of sarcasm detection. While the current graph density was a limitation, the results provide a strong foundation for future work. The authors highlight ethical risks regarding digital stereotyping and suggest that future research should focus on modeling recipient profiles to better predict sarcasm perception.

Summary updated

Abstract

Existing sarcasm detection systems focus on exploiting linguistic markers, context, or user-level priors. However, social studies suggest that the relationship between the author and the audience can be equally relevant for the sarcasm usage and interpretation. In this work, we propose a framework jointly leveraging (1) a user context from their historical tweets together with (2) the social information from a user’s neighborhood in an interaction graph, to contextualize the interpretation of the post. We distinguish between perceived and self-reported sarcasm identification. We use graph attention networks (GAT) over users and tweets in a conversation thread, combined with various dense user history representations. Apart from achieving state-of-the-art results on the recently published dataset of 19k Twitter users with 30K labeled tweets, adding 10M unlabeled tweets as context, our experiments indicate that the graph network contributes to interpreting the sarcastic intentions of the author more than to predicting the sarcasm perception by others.
BibTeX
@inproceedings{plepi2021perceived,
  author = {Joan Plepi and Lucie Flek},
  title = {{Perceived and Intended Sarcasm Detection with Graph Attention Networks}},
  booktitle = {Conference on Empirical Methods in Natural Language Processing},
  year = {2021},
  url = {https://openreview.net/forum?id=70S08KAGQ6},
}
Figure