Semantic Congruency Facilitates Memory for Emojis

Andriana L. Christofalos, Laurie Beth Feldman, Heather SheridanNAACL 2022

TL;DR

This study investigates how emojis, when integrated into text, affect our memory for them. The researchers found that emojis semantically related to the text (either as synonyms or inferences) are better remembered than unrelated emojis, suggesting that we process and store emoji meaning alongside text. This work highlights the growing importance of understanding emoji's role in higher-level cognitive processes beyond simple comprehension.

What Problem Does This Paper Address?

As emojis become ubiquitous in digital communication, understanding their cognitive impact is crucial. While previous research has explored how emojis influence immediate text comprehension and processing fluency, less is known about their effect on memory. Specifically, this paper addresses how the function of an emoji within a sentence—whether it directly replaces a word's meaning (synonym) or contributes to a broader inference—impacts how well we remember that emoji later. The authors aim to bridge the gap between immediate processing effects and longer-term memory retention for emojis, considering them not just as decorative elements but as active contributors to meaning.

Methods

The study employed a two-part experimental design:

  1. Ratings Task: Participants were presented with short passages (two sentences each) that concluded with an emoji. They then rated how well the emoji "fit" with the passage on a five-point scale. This task aimed to measure the perceived semantic coherence between the emoji and the text.
  2. Recognition Memory Task: Following the ratings task, participants completed a recognition test where they were shown a series of emojis and asked to identify whether they had seen them before in the context of the passages. This measured their ability to recall the emojis.

The researchers created stimuli with three emoji conditions:

  • Congruent Inference: The emoji supported an inference drawn from the passage.
  • Congruent Synonym: The emoji directly represented the meaning of a target word in the passage.
  • Incongruent: The emoji was unrelated to the passage's content.

Datasets

The study utilized newly created stimuli. Sixty short passages were developed, each consisting of two sentences and an emoji. These passages were designed to include cues for inference and elaboration on a target word. Each passage was paired with three types of non-emotion, object emojis: congruent inference, congruent synonym, and incongruent. No emoji was used in more than one sentence context.

Key Insights and Ideas

  • Emojis are not just decoration: The findings strongly suggest that readers actively process the semantic content of emojis and integrate it with the surrounding text, influencing memory retention.
  • Semantic relevance matters for memory: Emojis that are semantically connected to the text (both synonyms and inferences) are significantly better remembered than those that are unrelated. This aligns with the "levels of processing" framework, where deeper semantic engagement leads to stronger memory traces.
  • Synonym emojis are perceived as more coherent: While both synonym and inference emojis were rated as more coherent than incongruent ones, synonym emojis were judged to fit the passage slightly better than inference emojis. This might indicate a preference for direct semantic mapping over inferential contributions in terms of perceived coherence.
  • Inference emojis show a memory advantage (numerically): Although not statistically significant, inference emojis showed a trend towards higher recognition accuracy than synonym emojis. This hints that the cognitive effort involved in inferring meaning might lead to even stronger memory traces, a point the authors explore for future work.

Key Results

  • Coherence Ratings: Participants rated congruent inference emojis (mean fit = 3.12) and congruent synonym emojis (mean fit = 3.56) as significantly more coherent than incongruent emojis (mean fit = 1.30). Synonym emojis were rated as slightly more coherent than inference emojis.
  • Recognition Accuracy: Recognition accuracy was significantly higher for congruent emojis (inference: 90%, synonym: 86%) compared to incongruent emojis (82%). Specifically, inference emojis were better recognized than incongruent ones, and synonym emojis were also better recognized than incongruent ones.

Limitations and Future Work

  • Emoji Position: The study placed emojis exclusively at the end of passages. The authors acknowledge that emoji position might influence memory, suggesting future work should explore how different emoji placements (e.g., closer to relevant text) interact with their function.
  • Retention Interval: The memory task involved a short delay. Future research could investigate longer retention intervals to see if the observed memory effects persist or become more pronounced.
  • Emoji Discrimination: The study used a relatively straightforward recognition task. More challenging memory tasks with finer-grained emoji discrimination could potentially reveal stronger effects.
  • Inferential vs. Synonym Trade-off: While inference emojis showed a numerical advantage in memory, the difference wasn't statistically significant. Further research could explore this potential trade-off between the depth of semantic processing (inference) and direct semantic mapping (synonym) on memory recall.

Summary updated

BibTeX
@inproceedings{christofalos2022semantic,
  author = {Andriana L. Christofalos and Laurie Beth Feldman and Heather Sheridan},
  title = {{Semantic Congruency Facilitates Memory for Emojis}},
  booktitle = {Conference of the North American Chapter of the Association for Computational Linguistics},
  year = {2022},
}
Figure