TL;DR
This paper introduces SentEMO, a novel multilingual platform for fine-grained aspect-based sentiment and emotion analysis. It addresses the need for companies to gain deeper, more transparent insights from customer feedback by offering an adaptive system that can be fine-tuned with user data, improving over time.
What Problem Does This Paper Address?
In today's data-driven world, businesses are awash in customer feedback from reviews, emails, and social media. While sentiment analysis tools exist, they often operate as "black boxes," providing overall sentiment without detailing what aspects of a product or service are driving that sentiment, or how customers are feeling (e.g., angry, joyful). Furthermore, many existing tools are language-specific and lack adaptability to a company's unique data. SentEMO aims to bridge this gap by providing a multilingual, aspect-aware system that not only identifies sentiment polarity but also detects specific emotions associated with particular aspects, all within an interactive and customizable platform.
Methods
SentEMO employs a pipeline approach, where the output of one module feeds into the next. The core of the system involves several key components:
- Aspect Term Extraction (ATE): This initial step identifies explicit mentions of aspects or features within a text. The authors utilize a Conditional Random Field (CRF) model, enhanced with token-shape features, linguistic information, and dependency parsing.
- Aspect Category Classification (ACC): Here, extracted aspect terms are mapped to predefined categories (e.g., "ROOM_style," "PERSONNEL_friendliness"). A Support Vector Machine (SVM) classifier is used, leveraging lexico-semantic features and Word2Vec embeddings.
- Aspect Polarity Classification (APC): This module determines the sentiment (very positive, positive, neutral, negative, very negative) associated with each identified aspect. A pre-trained Dutch BERT model (RobBERT) is fine-tuned and used with a linear SVM.
- Aspect-Based Emotion Analysis (ABEA): Building upon sentiment analysis, this component adds an emotion layer, classifying emotions like anger, joy, sadness, etc., associated with specific aspects. Similar to APC, an SVM classifier with RobBERT features is employed, but with a larger context window.
A key methodological innovation is the platform's adaptability: users can correct or label data and retrain the system, creating a feedback loop that continuously improves its performance on company-specific data.
Datasets
The SentEMO project utilizes data collected from eight Belgian companies across six domains: FMCG (Fast-Moving Consumer Goods), Airline, Hotel, Product Retail, Hospital, and Telecom. For the in-house Dutch data, over 1,000 reviews were scraped for FMCG, Airline, and Hotel domains from sources like bol.com, Trustpilot, and Tripadvisor. Data for other domains was provided by project partners. After initial cleaning, datasets comprised at least 900 instances per domain. This data was manually annotated with aspect terms, categories, polarities, and emotions, forming a valuable benchmark for the system.
Key Insights and Ideas
- The Power of Granularity: Moving beyond overall sentiment to aspect-level analysis reveals much richer insights. For instance, a product might have a "great location" (positive) but a "terrible price" (negative), a nuance lost in a general sentiment score.
- Emotion Matters: Understanding the specific emotions (anger, joy, satisfaction) behind sentiment is crucial for effective customer relationship management and targeted responses.
- Adaptability is Key: A one-size-fits-all approach to sentiment analysis is insufficient. The ability for users to fine-tune models with their own data is a significant step towards practical, industry-ready tools.
- Implicit Aspects are Detectable: The system can infer sentiment and aspects even when they are not explicitly stated, demonstrating a sophisticated understanding of language.
Key Results
The paper reports strong performance across various sub-tasks and domains. For Aspect Term Extraction, micro-averaged F1-scores reached up to 94.3% for the General domain. Aspect Category Classification achieved accuracies up to 86.6%. Aspect Polarity Classification saw F1-scores up to 89.5%. Emotion Classification, while more complex, also demonstrated promising results, with cost-corrected accuracy being particularly high for the Hospital domain. The interactive dashboards provide clear visualizations of these results, allowing users to explore sentiment and emotion at the aspect level.
Limitations and Future Work
The authors acknowledge that performance is not "perfect" and is influenced by the limited training data available for some tasks and the complexity of the five-way classification for polarity. Potential error percolation between pipeline stages is also a consideration.
Future work includes expanding the platform to support English, French, and German, leveraging existing BERT models for these languages. A key future direction is to implement an intuitive annotation interface for users to flag incorrect predictions, suggest corrections, and ultimately retrain the models, further enhancing the platform's adaptability and accuracy.
Summary updated