TL;DR
This paper introduces MoodSwipe, a novel soft keyboard that helps users express emotions more effectively in text messages. By analyzing user input in real-time and allowing users to specify their desired emotion, MoodSwipe suggests contextually relevant messages, bridging the gap between text communication and emotional intelligence.
What Problem Does This Paper Address?
In our increasingly text-based communication world, conveying and understanding emotions accurately can be challenging. While technologies exist to classify emotions in text, they are often used to help recipients understand messages, not to assist senders in expressing their own emotions effectively. This leaves a gap for users who struggle to find the right words to convey their feelings, whether they want to soften an angry message, express gratitude, or even deliberately inject humor or assertiveness. Existing solutions often rely on visual cues or typing patterns, which are secondary to the text itself, or require external hardware for more nuanced emotion detection. MoodSwipe aims to solve this by providing a seamless, integrated solution within the keyboard itself.
Methods
MoodSwipe functions as a soft keyboard that integrates emotion classification and text suggestion. Here's how it works:
- Real-time Emotion Detection: As a user types, MoodSwipe analyzes the input text to detect its emotion in real-time.
- Color-Coded Emotion Bar: A color bar above the keyboard visually represents the detected emotion, with different colors corresponding to seven predefined emotions (Anger, Joy, Sadness, Fear, Anticipation, Tired, and Neutral).
- User-Specified Emotion Selection: Users can "swipe" along the color bar to manually select or adjust the emotion they wish to convey. This is a key feature, allowing users to override the system's detection or explicitly state their intended emotion.
- Contextual Message Suggestions: Based on the detected or user-specified emotion and the input text, MoodSwipe suggests relevant pre-written messages. The act of viewing these suggestions serves as an incentive for users to refine their emotion detection.
- Data Collection: The system logs user interactions, including typed text, selected suggestions, and emotion labels, to facilitate the collection of valuable, self-reported emotion labels for dialog messages.
The emotion classification models used are based on CNN (Convolutional Neural Network) and LSTM (Long Short-Term Memory) architectures, trained on a combination of blog data, tweet data, and their own collected dialog data. For text suggestion, they employ a retrieval-based model using Lucene, enhanced with an emotion-filtering mechanism.
Datasets
The paper utilizes several datasets:
- EmotionPush Dataset: A newly collected dataset of 162,031 social dialog messages, which forms the core of their training and evaluation data. A subset of 8,818 messages was manually labeled for emotions.
- LJ40K: A dataset containing 40 emotions, mapped to the seven emotions used in MoodSwipe.
- Tweet Data: Tweets collected via Twitter streaming API, filtered by hashtags related to the seven emotions.
Key Insights and Ideas
- Bridging the Sender-Receiver Emotion Gap: MoodSwipe uniquely focuses on empowering the sender to express emotions, rather than just helping the receiver interpret them.
- User-Specified Emotion as a Powerful Context: Allowing users to explicitly select their desired emotion significantly enhances the relevance and utility of suggested messages. This provides a more direct and active way for users to guide the system.
- Incentivizing Data Labeling: The interface design, where viewing suggestions encourages users to correct emotion labels, offers a natural and less intrusive way to collect valuable self-reported emotion data for dialogs.
- Emotion as a Filter for Text Suggestion: Incorporating emotion as a constraint in retrieval-based text suggestion models demonstrably improves the quality and relevance of the suggestions.
Key Results
- Emotion Classification Performance: CNN and LSTM models achieved high accuracies (over 0.9) for the three major emotions (Joy, Anger, Sadness) when trained on their dialog data, validating its suitability for this task. CNN was chosen as the final model for MoodSwipe due to its efficiency.
- Text Suggestion Quality: Human evaluations showed that MoodSwipe's suggested texts were considered "good" (better than the user's original response) in 26-28% of cases. The [+Emotion] setting, which uses user-specified emotions, performed slightly better than the baseline (without explicit emotion specification).
- Anger Expression Benefit: The "Good Suggestion Rate" for comfort in messages related to Anger was notably high (around 40%), suggesting MoodSwipe is particularly effective in assisting users to express anger appropriately.
Limitations and Future Work
- Retrieval-Based Model Limitations: The current retrieval-based model, even with emotion filtering, has limitations. The authors acknowledge that the performance can be improved by increasing the size and diversity of the dialog database.
- Automatic Annotation: The emotions in the suggestion database are automatically annotated, which may introduce noise compared to human labels.
- Future Directions: The authors envision that with more data and more sophisticated models, the benefits of incorporating emotion context can be further boosted. They also highlight the potential for MoodSwipe to collect more user-reported labels, leading to a more comprehensive emotion-sensitive communication framework. The system is available on Google Play, suggesting a path towards wider adoption and further data collection.
Summary updated