Abstract
Multimodal Aspect-Based Sentiment Analysis (MABSA) aims to identify specific aspects and their associated sentiments within combined text and image data. Current models often struggle in complex scenarios involving multiple aspects and conflicting sentiments. This paper introduces the Aspect Enhancement and Text Simplification (AETS) framework. AETS addresses aspect neglect through a dynamic masking mechanism and mitigates sentiment misalignment via a syntactic dependency-based text simplification module. Experimental results on benchmark datasets demonstrate that AETS achieves state-of-the-art performance, particularly in noisy social media environments.
Introduction
The task of MABSA is critical for understanding nuanced opinions in social media. However, existing approaches frequently encounter two primary obstacles: aspect neglect, where models fail to prioritize relevant terms within a multimodal context, and sentiment misalignment, where models incorrectly pair adjectives with subjects in multi-aspect sentences. For instance, in a post containing a photo of a restaurant and the text, "The burger was amazing, but the wait time was a nightmare," a standard model might struggle to isolate the positive sentiment for the food from the negative sentiment for the service. The AETS framework is designed to resolve these ambiguities by isolating core semantic relationships.
The AETS Framework
The proposed AETS framework enhances standard backbones (RoBERTa for text and ViT for images) through two innovative modules designed to reduce noise and focus on core semantic relationships.
Aspect Enhancement Module
To prevent the model from overlooking key subjects, AETS utilizes the spaCy NLP library to identify candidate aspect terms (nouns). A dynamic masking vector is then generated to filter hidden states across both text and image modalities. This "spotlight" effect ensures that the model's internal representations are highly sensitive to potential aspects before the final classification stage, effectively adding enhanced features back to the original representations.
Text Simplification Module
The core of the AETS noise-reduction strategy is the Text Simplification module. Rather than processing raw, often rambling sentences, the module constructs a syntactic dependency graph for the input text to identify the most relevant sentiment descriptors.
The process involves:
- Dependency Graph Construction: Mapping the grammatical relationships between words.
- Breadth-First Search (BFS): Starting from a target aspect, the algorithm traverses the graph to locate the most closely related sentiment words (adjectives).
- Syntactic Pruning: Irrelevant words are discarded, reducing the "interference" from unrelated descriptors.
- Quicksort Reassembly: The remaining core words are reordered into a simplified, coherent phrase (e.g., "The food delicious") which is then concatenated with the original input to provide a clear signal for the classifier.
Experimental Evaluation
The AETS model was evaluated on the Twitter-2015 and Twitter-2017 benchmark datasets. These datasets are characterized by the informal nature of social media language and the frequent presence of multiple aspects within a single post.
Key Results
AETS achieved state-of-the-art (SOTA) results across three primary sub-tasks:
- Joint Analysis (JMASA): Achieved F1 scores of 72.1% (Twitter-2015) and 73.1% (Twitter-2017), outperforming previous leaders like Atlantis and AoM.
- Aspect Extraction (MATE): Showed significant performance gains with F1 scores reaching 90.9% and 95.3%, validating the effectiveness of the Aspect Enhancement module.
- Sentiment Classification (MASC): Maintained top-tier performance, proving that the simplified text provides a much cleaner signal for determining sentiment.
Discussion and Conclusion
The AETS framework demonstrates that "less is more" in multimodal sentiment analysis. By physically pruning syntactically irrelevant text and highlighting key aspects through dynamic masking, the model significantly reduces interference in complex multi-sentiment scenarios. Ablation studies confirmed that the multimodal synergy is essential; removing the image component led to a significant drop in performance, proving visual context is vital for disambiguation. While the advantage is most pronounced in cluttered sentences, future work may extend these simplification principles to the visual modality, using object detection to further refine the multimodal signal.
Summary updated