Abstract
Emoji is an essential component in dialogues which has been broadly utilized on almost all social platforms. It could express more delicate feelings beyond plain texts and thus smooth the communications between users, making dialogue systems more anthropomorphic and vivid. In this paper, we focus on automatically recommending appropriate emojis given the contextual information in multi-turn dialogue systems, where the challenges locate in understanding the whole conversations. More specifically, we propose the hierarchical long short-term memory model (H-LSTM) to construct dialogue representations, followed by a softmax classifier for emoji classification. We evaluate our models on the task of emoji classification in a real-world dataset, with some further explorations on parameter sensitivity and case study. Experimental results demonstrate that our method achieves the best performances on all evaluation metrics. It indicates that our method could well capture the contextual information and emotion flow in dialogues, which is significant for emoji recommendation.
Introduction
While emojis are a staple of modern digital communication, teaching a machine to use them correctly is surprisingly difficult. Most existing systems treat emoji recommendation as a simple text classification task, looking only at the sentence currently being written. However, human conversation is deeply contextual.
As shown in the figure above, a "delicious" emoji might be perfect for a reply like "That's a great idea!", even though the word "food" never appears in that specific sentence. The context—an invitation to eat—lives in the previous turns of the dialogue. The challenge lies in effectively modeling this multi-turn contextual information to predict the most fitting emoji for a given reply.
Methodology
The proposed Hierarchical Long Short-Term Memory (H-LSTM) architecture operates on two distinct levels to preserve the natural structure of a conversation:
- Word-level LSTM: This layer processes each individual sentence in the dialogue turn-by-turn, encoding them into fixed-length vector representations.
- Sentence-level LSTM: This higher-level layer takes the sequence of sentence representations and processes them to build a global "dialogue representation."
This hierarchical approach allows the model to track the "emotion flow" across an entire conversation. The final dialogue representation is fed into a softmax classifier to predict the probability of various emoji candidates. The authors compared this against two baselines: S-LSTM (which only looks at the reply sentence) and F-LSTM (which flattens the entire dialogue into one long string of words).
Experimental Setup
The models were trained and evaluated on the Weibo2015 dataset, a large-scale collection of Chinese social media dialogues containing approximately 1.16 million training instances. The authors selected the 10 most frequent emojis to serve as classification labels. To manage complexity, dialogues were limited to a maximum of 4 turns, and individual sentences were capped at 50 words.
Key Results
The H-LSTM model achieved the best performance across all evaluation metrics, including Precision at 1 (P@1), Precision at 3 (P@3), and Mean Reciprocal Rank (MRR).
- P@1 Accuracy: H-LSTM reached 35.4%, compared to 32.5% for the single-sentence LSTM and 29.6% for a traditional Bag-of-Words baseline.
- Top-3 Accuracy: When looking at the top 3 recommendations, H-LSTM achieved 65.7% accuracy.
- Parameter Sensitivity: Performance generally improved as the dimensions of word embeddings and hidden states increased, peaking at around 384 dimensions before stabilizing.
Discussion and Insights
Experimental analysis confirmed that context is non-negotiable; models incorporating multi-turn context (F-LSTM and H-LSTM) consistently outperformed the single-sentence model (S-LSTM). Furthermore, the H-LSTM outperformed the "flattened" F-LSTM, proving that simply concatenating sentences loses the hierarchical relationships that define human dialogue.
Case studies revealed that simpler models often fell into "keyword traps." For example, an S-LSTM might suggest a food-related emoji just because a specific food item was mentioned in the reply, even if the overall sentiment was negative. The H-LSTM was better at avoiding these local traps by maintaining a global state of the conversation.
Conclusion and Future Work
While H-LSTM significantly improves emoji recommendation, challenges remain regarding emoji interchangeability (semantically similar emojis) and personalization. Future work aims to incorporate user-specific preferences and more flexible emoji positioning to make dialogue systems feel even more natural and anthropomorphic.
Summary updated