Why We Feel What We Feel: Joint Detection of Emotions and Their Opinion Triggers in E-commerce

Arnav Attri, Anuj Attri, Pushpak Bhattacharyya, Suman Banerjee, Amey Patil, Muthusamy Chelliah, Nikesh GareraEMNLP 2025

TL;DR

This paper introduces EOT (Emotion-Opinion Trigger) detection, a novel joint task for e-commerce reviews that identifies not only emotions but also the specific text spans (opinion triggers) that caused them. They developed EOT-X, a new benchmark dataset, and EOT-DETECT, a structured prompting framework for LLMs, demonstrating significant improvements over existing methods.

What Problem Does This Paper Address?

Understanding customer sentiment in e-commerce reviews is crucial for businesses. While emotion detection has been studied, a significant gap exists in understanding why customers feel a certain way. Simply knowing a review expresses "disgust" is less informative than knowing what specifically in the product or service caused that disgust (e.g., finding hair in a makeup product). Existing research hasn't jointly tackled emotion detection and the extraction of these explanatory "opinion triggers" in e-commerce contexts, hindering a deeper, actionable understanding of customer feedback.

Methods

The authors propose a novel joint task called EOT (Emotion-Opinion Trigger) detection. This task aims to:

  1. Emotion Identification: Detect emotions from Plutchik's 8 primary emotions (plus Neutral).
  2. Opinion Trigger Extraction: Extract exact text spans from the review that explain the identified emotion.

To achieve this, they developed EOT-DETECT, a structured prompting framework for Large Language Models (LLMs). EOT-DETECT uses a multi-step approach:

  • System Message: Defines the LLM's role as an expert.
  • Task Description: Outlines the task, defines opinion triggers, and specifies the emotion scope.
  • 5-Step Reasoning Instructions: Guides the LLM through identifying emotions, linking them to triggers, maintaining balance, ensuring clarity, and adhering to extractive constraints.
  • Final Self-Check: A crucial step where the LLM verifies emotion coverage, trigger coverage, faithfulness, and verifiability before outputting the result.

They also fine-tuned a compact LLM, EOT-LLaMA, on their dataset.

Datasets

  • EOT-X: A newly introduced, human-annotated benchmark dataset consisting of 2,400 e-commerce reviews from Amazon, TripAdvisor, and Yelp. Each review is labeled with fine-grained emotions (Plutchik's 8 primary emotions) and extractive opinion triggers. This dataset is designed to support cross-domain generalization.
  • Aggregated Gold Standard: For evaluation, they created an aggregated gold standard by combining annotations from three expert raters, using majority voting for emotions and preserving all unique triggers.

Key Insights and Ideas

  • The "Why" Matters: Understanding the specific triggers behind emotions is far more valuable than just knowing the emotion itself for actionable insights.
  • Plutchik's Framework is Optimal: The authors argue that Plutchik's 8 primary emotions provide a balanced and taxonomically constrained yet expressive set for emotion analysis, preventing "emotion explosions" from LLMs.
  • Structured Prompting is Key: EOT-DETECT significantly outperforms standard zero-shot and chain-of-thought prompting, highlighting the importance of carefully designed prompts with explicit reasoning and self-reflection steps for complex NLP tasks.
  • Compact Models Can Compete: Fine-tuned smaller models (like EOT-LLaMA) can achieve competitive performance, making advanced emotion analysis more accessible and deployable on consumer-tier hardware.
  • LLM Behavior Varies: Different LLM families and sizes exhibit distinct performance characteristics, with larger models generally performing better, but architectural design and training strategies also play a critical role.

Key Results

  • EOT-DETECT Superiority: The EOT-DETECT framework achieved superior performance compared to zero-shot and zero-shot chain-of-thought methods across various LLMs. For example, Mistral-7B-Instruct-v0.2 with EOT-DETECT achieved the highest emotion metrics (P: 0.86, R: 0.87, F1: 0.86).
  • Closed-Source vs. Open-Source: Closed-source models like Claude 3.5 Sonnet with EOT-DETECT achieved the highest absolute scores (P: 0.94, R: 0.72, F1: 0.81), but state-of-the-art open-source models are rapidly closing the gap.
  • Fine-Tuned Models: Fine-tuned models showed improved consistency, with EOT-LLaMA (a 1B-parameter model) outperforming larger counterparts in some aspects.
  • High Inter-Annotator Agreement: Expert annotators achieved high agreement scores (emotion κ = 0.88, trigger κ = 0.84) on the EOT-X dataset, indicating the task's feasibility and the robustness of their annotation guidelines.

Limitations and Future Work

  • Dataset Size: The EOT-X dataset, while comprehensive, is moderately sized due to the labor-intensive nature of annotating fine-grained emotions and triggers.
  • LLM Opacity: The authors could not test state-of-the-art OpenAI models' internal reasoning steps due to their proprietary nature, limiting deeper analysis of their CoT processes.
  • Hallucinations: LLMs, even with EOT-DETECT, may still produce hallucinations or unintended outputs, especially in complex cases.
  • Future Work: The authors suggest further research into more sophisticated aggregation methodologies, exploring LLM architectures for better reasoning, and expanding the dataset. They also highlight the need for careful integration of these models into real-world applications, acknowledging LLMs do not experience emotions themselves.

Summary updated

BibTeX
@inproceedings{attri2025why,
  author = {Arnav Attri and Anuj Attri and Pushpak Bhattacharyya and Suman Banerjee and Amey Patil and Muthusamy Chelliah and Nikesh Garera},
  title = {{Why We Feel What We Feel: Joint Detection of Emotions and Their Opinion Triggers in E-commerce}},
  booktitle = {Conference on Empirical Methods in Natural Language Processing},
  year = {2025},
}
Figure