Abstract
ECoK (Emotional Commonsense Knowledge Graph) is a large-scale dataset and model designed to bridge the gap in AI's understanding of human emotional reasoning. By structuring over 144,000 high-quality emotional reasoning triples across seven distinct dimensions—including bodily symptoms, action tendencies, and tailored advice—ECoK provides a framework for interpretable emotional intelligence. Experimental results demonstrate that a specialized generator trained on this data, COMET-ECoK, outperforms general-purpose models like GPT-4-Turbo in emotional reasoning and significantly enhances performance in downstream tasks like emotion recognition.
Introduction
While Large Language Models (LLMs) excel at many tasks, they often lack a deep, interpretable understanding of human commonsense regarding emotions. Existing knowledge graphs (KGs) like ConceptNet or ATOMIC focus primarily on factual or social sequences, often treating emotions as shallow attributes. However, emotion is a multifaceted psychological phenomenon involving cognition, physiology, and behavior. ECoK aims to fill this "emotional gap" by providing a structured framework for machines to mine "emotional gold" from text, reasoning about unobserved causes, physical sensations, and social context.
Methodology
The construction of ECoK followed a sophisticated pipeline combining automated mining with human oversight to ensure data quality and relevance.
- Event Selection: Researchers extracted 4.9K unique "head entities" from ATOMIC20/20 and Twitter sentiment datasets. BERT-based clustering was used to merge similar events and filter noise.
- Attribute Acquisition: Using Llama-2-70B, the team generated "tail entities" across seven specific relations:
group,causes,bodily symptoms,feelings,expression,action tendencies, andadvice. - Context-Aware Advice: Notably, the
advicerelation was generated by feeding the model both the event and the targetgroup, ensuring recommendations were personalized (e.g., advice for a "YouTuber" differs from that for a "student"). - Human Examine: Experts manually inspected batches of generated triples to filter for relevance, safety, and authenticity.
- COMET-ECoK: A BART-based knowledge generator was trained on this data, allowing the model to produce emotional commonsense for entirely new, unseen events.
The ECoK Framework
Rather than simply labeling an emotion, ECoK deconstructs it into a seven-dimension framework that mirrors psychological "component process models." This approach asks: Who is feeling it? What caused it? What does their body feel like? How do they act?
The resulting graph contains 144,005 triples covering 60,392 distinctive "tails." This structure allows models to specialize in emotional reasoning rather than relying on general linguistic patterns.
Key Results
- Superior Generation: In automatic evaluations, COMET-ECoK consistently beat GPT-4-Turbo, achieving a BLEU-4 score of 79.83 compared to GPT-4-Turbo's 72.87.
- Human Approval: Human evaluators accepted 90.26% of the inferences generated by COMET-ECoK, the highest among all tested models (including Llama-2 and GPT-4 variants).
- Downstream Gains: Integrating ECoK as an external knowledge source led to new state-of-the-art results. On the MELD dataset for Emotion Recognition in Conversation (ERC), the ECoK-enhanced model achieved a Weighted F1 of 66.30, outperforming versions using ConceptNet or ATOMIC.
Conclusion
ECoK demonstrates that smaller, specialized models trained on high-quality structured data can outperform massive generalist models in specific reasoning tasks. While the graph may inherit some biases from its social media source data, it provides a robust foundation for more emotionally aware human-machine interaction. Future work will look toward even more fine-grained emotional representations and broader generalization across real-world applications.
Summary updated