ECoK: Emotional Commonsense Knowledge Graph for Mining Emotional Gold

Zhunheng Wang, Xiaoyi Liu, Mengting Hu, Rui Ying, Ming Jiang, Jianfeng Wu, Yalan Xie, Hang Gao, Renhong ChengACL 2024 · DOI

Abstract

ECoK (Emotional Commonsense Knowledge Graph) is a large-scale dataset and model designed to bridge the gap in AI's understanding of human emotional reasoning. By structuring over 144,000 high-quality emotional reasoning triples across seven distinct dimensions—including bodily symptoms, action tendencies, and tailored advice—ECoK provides a framework for interpretable emotional intelligence. Experimental results demonstrate that a specialized generator trained on this data, COMET-ECoK, outperforms general-purpose models like GPT-4-Turbo in emotional reasoning and significantly enhances performance in downstream tasks like emotion recognition.

Introduction

While Large Language Models (LLMs) excel at many tasks, they often lack a deep, interpretable understanding of human commonsense regarding emotions. Existing knowledge graphs (KGs) like ConceptNet or ATOMIC focus primarily on factual or social sequences, often treating emotions as shallow attributes. However, emotion is a multifaceted psychological phenomenon involving cognition, physiology, and behavior. ECoK aims to fill this "emotional gap" by providing a structured framework for machines to mine "emotional gold" from text, reasoning about unobserved causes, physical sensations, and social context.

Methodology

The construction of ECoK followed a sophisticated pipeline combining automated mining with human oversight to ensure data quality and relevance.

The ECoK construction pipeline, illustrating the flow from event selection and cluster analysis to human examination.
  1. Event Selection: Researchers extracted 4.9K unique "head entities" from ATOMIC20/20 and Twitter sentiment datasets. BERT-based clustering was used to merge similar events and filter noise.
  2. Attribute Acquisition: Using Llama-2-70B, the team generated "tail entities" across seven specific relations: group, causes, bodily symptoms, feelings, expression, action tendencies, and advice.
  3. Context-Aware Advice: Notably, the advice relation was generated by feeding the model both the event and the target group, ensuring recommendations were personalized (e.g., advice for a "YouTuber" differs from that for a "student").
  4. Human Examine: Experts manually inspected batches of generated triples to filter for relevance, safety, and authenticity.
  5. COMET-ECoK: A BART-based knowledge generator was trained on this data, allowing the model to produce emotional commonsense for entirely new, unseen events.

The ECoK Framework

Rather than simply labeling an emotion, ECoK deconstructs it into a seven-dimension framework that mirrors psychological "component process models." This approach asks: Who is feeling it? What caused it? What does their body feel like? How do they act?

An example of the ECoK graph structure showing interconnected emotional dimensions for a protagonist in various scenarios.

The resulting graph contains 144,005 triples covering 60,392 distinctive "tails." This structure allows models to specialize in emotional reasoning rather than relying on general linguistic patterns.

Key Results

  • Superior Generation: In automatic evaluations, COMET-ECoK consistently beat GPT-4-Turbo, achieving a BLEU-4 score of 79.83 compared to GPT-4-Turbo's 72.87.
  • Human Approval: Human evaluators accepted 90.26% of the inferences generated by COMET-ECoK, the highest among all tested models (including Llama-2 and GPT-4 variants).
  • Downstream Gains: Integrating ECoK as an external knowledge source led to new state-of-the-art results. On the MELD dataset for Emotion Recognition in Conversation (ERC), the ECoK-enhanced model achieved a Weighted F1 of 66.30, outperforming versions using ConceptNet or ATOMIC.

Conclusion

ECoK demonstrates that smaller, specialized models trained on high-quality structured data can outperform massive generalist models in specific reasoning tasks. While the graph may inherit some biases from its social media source data, it provides a robust foundation for more emotionally aware human-machine interaction. Future work will look toward even more fine-grained emotional representations and broader generalization across real-world applications.

Summary updated

BibTeX
@inproceedings{wang2024ecok,
  author = {Zhunheng Wang and Xiaoyi Liu and Mengting Hu and Rui Ying and Ming Jiang and Jianfeng Wu and Yalan Xie and Hang Gao and Renhong Cheng},
  title = {{ECoK: Emotional Commonsense Knowledge Graph for Mining Emotional Gold}},
  booktitle = {Annual Meeting of the Association for Computational Linguistics},
  year = {2024},
  doi = {10.18653/v1/2024.findings-acl.480},
}
Figure