TL;DR
This paper introduces SarcOji, a novel dataset of over 29,000 English texts with emojis, specifically curated to study the role of emojis in sarcasm detection. By analyzing this dataset, the authors found that emoji intensity and position are strong indicators of sarcasm, even when text and emoji polarities don't directly align.
What Problem Does This Paper Address?
Identifying sarcasm in text is a notoriously difficult task for computational models due to its reliance on context and subtle linguistic cues. While many approaches exist, from rule-based systems to deep learning, they often struggle with the nuances of human communication. Emojis, increasingly prevalent in online discourse, are recognized as potential indicators of sentiment and intent, including sarcasm. However, a significant gap exists in the availability of comprehensive datasets that specifically link emojis to sarcasm, making it challenging to thoroughly investigate their role. This paper aims to bridge this gap by creating and analyzing SarcOji, a dataset designed to facilitate the study of emoji-driven sarcasm.
Methods
The core of this work lies in the creation and analysis of the SarcOji dataset. The authors compiled data from five existing sarcasm datasets, filtering for texts that contained emojis. They then performed several preprocessing steps, including language filtering, removal of URLs and hashtags, and ensuring all included texts had emojis.
To analyze the data, they employed various sentiment analysis tools (SentiWordNet, VADER, TextBlob) to derive sentiment scores for both the text and the emojis within it. They also developed a method to identify the most frequent emoji in a text and its position (start, middle, or end). This allowed them to investigate potential incongruences between text and emoji sentiment, as well as the impact of emoji frequency and position on sarcasm detection.
Datasets
- SarcOji: A newly compiled dataset consisting of 29,377 labeled records with emojis, derived from five publicly available sarcasm datasets.
- Sarcasm Dataset harvested from Twitter by Ghosh and Veale (2016)
- Dataset compiled by Subramanian et al. (2019)
- Oprea and Magdy (2019) dataset
- Shared Task on Sarcasm (Twitter and Reddit) dataset at FigLang'2020 (Ghosh et al., 2020)
- Intended Sarcasm Dataset in English from iSarcasmEval Task at SemEval'22 (Abu Farha et al., 2022)
Key Insights and Ideas
- Emoji Intensity Matters: The frequency with which an emoji is used within a text, particularly when used repeatedly (intensity), appears to be a strong indicator of sarcasm. Sarcastic texts showed a higher intensity of emoji usage compared to non-sarcastic texts.
- Position is Key: The placement of emojis within a text is also significant. While emojis in non-sarcastic texts tend to appear at the end, in sarcastic texts, they are more frequently found closer to the context, suggesting a more integrated role in conveying the sarcastic intent.
- Incongruence is Subtle: While the study looked for direct polarity mismatches between text and emojis, significant incongruence wasn't a dominant finding. This suggests that sarcasm conveyed through emojis might be more nuanced than a simple positive-negative clash.
- Neutrality is Important: The prevalence of neutral texts and emojis in both sarcastic and non-sarcastic categories highlights the need for further exploration into how these seemingly neutral elements contribute to sarcasm.
Key Results
The SarcOji dataset contains 11,448 sarcastic and 17,929 non-sarcastic texts with emojis. The study found that:
- The emoji "😂" (face with tears of joy) was the most frequently used emoji in sarcastic texts.
- While "😂" was common in both sarcastic and non-sarcastic texts, its intensity (repeated usage) was significantly higher in sarcastic contexts.
- In non-sarcastic texts, the most frequent emoji (MaxEmoji) appeared at the end of the text in 80% of cases. In contrast, for sarcastic texts, MaxEmoji appeared at the beginning 13.75% of the time, in the middle 18.9% of the time, and at the end 67.35% of the time.
- The paper reports percentages of incongruence between text and emoji sentiment scores across different tools, with values generally ranging from around 7% to 22% depending on the tool and whether all emojis or just the MaxEmoji were considered.
Limitations and Future Work
The authors acknowledge that sentiment analysis tools do not always agree on emoji sentiment scores, which can impact the determination of incongruence. They also note that current emoji resources may not fully capture the sentiment of newer emojis.
Future work suggested includes:
- Augmenting existing emoji resources to specifically flag sarcastic usage.
- Further investigation into the role of seemingly neutral texts and emojis in discerning sarcasm.
- Developing tools that can assign and regularly update numerical scores for emojis to better capture their evolving sentiment and contextual usage.
- Building a tool that can assign and regularly update numerical scores for emojis to identify the sentiment they express, as the same emoji can have different connotations depending on context and intensity.
Summary updated