Abstract
Document-Level Target Sentiment Analysis (DTSA) faces significant challenges in tracking sentiment across multiple sentences, particularly when entities are referenced through coreferential mentions such as pronouns or descriptive synonyms. This paper introduces a joint multi-task learning framework that explicitly models coreference resolution alongside sentiment classification. By employing a dual-path RoBERTa architecture with shared parameters and a novel token-level constraint, the proposed method effectively aggregates sentiment across entity clusters. Furthermore, the authors introduce a cluster-based evaluation metric that more accurately reflects model performance by accounting for valid coreferential links.
1. Introduction
While Aspect-Based Sentiment Analysis (ABSA) has seen significant progress at the sentence level, real-world reviews often span multiple sentences where context is distributed. A critical hurdle in Document-Level Target Sentiment Analysis (DTSA) is the presence of coreference—where different words (e.g., "shoes," "they," "the leather") refer to the same entity. Traditional models often fail to link these mentions, leading to fragmented or incorrect sentiment assignments.
2. Problem Statement: The Coreference Gap
In a typical review, a user might introduce a product feature and then refer back to it using pronouns. Standard models often treat these as separate entities or ignore the sentiment associated with pronouns entirely. This results in an incomplete understanding of the reviewer's intent.
As shown in Figure 1, traditional decoders might identify "leather" as positive and "toe area" as negative but fail to recognize that "they" (referring to the work shoes) carries a neutral or mixed overall sentiment. The proposed method (NTSA) explicitly identifies these coreferential mentions to build a unified entity representation.
3. Methodology
The researchers propose a dual-path framework based on RoBERTa, designed to learn sentiment extraction and coreference resolution simultaneously.
3.1 Dual-Path Architecture
The framework consists of two primary components as illustrated in Figure 2:
- DTSA Path: Responsible for extracting opinion targets, classifying sentiment (Positive, Negative, Mixed), and identifying hierarchical relationships (e.g., recognizing that "toe area" is a sub-component of "shoes").
- Coreference Path: Identifies mentions across the document that refer to the same underlying entity.
3.2 Key Innovations
- Joint Learning and Parameter Sharing: The two paths share underlying RoBERTa parameters, allowing the model to leverage coreference context for better sentiment understanding.
- Token-Level Coreference Constraint (TC): This module aligns the representations between the two paths, ensuring that the model's internal representation of an entity remains consistent across its various mentions (e.g., name vs. pronoun).
- Cluster-Based Metric: To address the limitations of strict string-matching metrics, the authors developed a "loose matching" approach. This metric grants credit if a model correctly identifies sentiment for any member of a coreference cluster, rather than requiring an exact match with a specific gold-standard mention.
4. Experimental Setup and Datasets
The model was evaluated on the DTSA dataset, covering four domains: Books, Clothing, Hotels, and Restaurants. To facilitate training, the authors augmented the dataset with coreference annotations from two sources:
- Human Experts: Manual clustering of entities.
- LLM-Assisted (ChatGPT): Using GPT-3.5-Turbo to generate clusters, which were found to be 92% accurate upon manual verification. This suggests that LLMs can significantly reduce the cost of complex NLP data annotation.
5. Results and Discussion
The proposed joint model achieved state-of-the-art performance with an average F1 score of 47.15%, outperforming the previous best-performing baseline (Seq2Struct*).
A significant finding was the impact of the evaluation metric. When moving from a strict metric to the new cluster-based metric, the model's score improved from 47.15% to 48.94%. This 1.8-point jump suggests that previous benchmarks have systematically underestimated model accuracy by penalizing valid coreferential resolutions. The improvements were most notable in the Book and Clothing domains, which typically feature more complex, multi-sentence descriptions.
6. Conclusion
This research demonstrates that explicitly modeling coreference is vital for document-level sentiment analysis. By integrating coreference resolution into a joint learning framework and adopting more flexible evaluation metrics, the researchers have provided a more robust and realistic approach to understanding complex consumer reviews. Future work will focus on reducing noise in specific domains and optimizing LLM-based annotation pipelines.
Summary updated