A Joint Coreference-Aware Approach to Document-Level Target Sentiment Analysis

Hongjie Cai, Heqing Ma, Jianfei Yu, Rui XiaACL 2024 · DOI

Abstract

Document-Level Target Sentiment Analysis (DTSA) faces significant challenges in tracking sentiment across multiple sentences, particularly when entities are referenced through coreferential mentions such as pronouns or descriptive synonyms. This paper introduces a joint multi-task learning framework that explicitly models coreference resolution alongside sentiment classification. By employing a dual-path RoBERTa architecture with shared parameters and a novel token-level constraint, the proposed method effectively aggregates sentiment across entity clusters. Furthermore, the authors introduce a cluster-based evaluation metric that more accurately reflects model performance by accounting for valid coreferential links.

1. Introduction

While Aspect-Based Sentiment Analysis (ABSA) has seen significant progress at the sentence level, real-world reviews often span multiple sentences where context is distributed. A critical hurdle in Document-Level Target Sentiment Analysis (DTSA) is the presence of coreference—where different words (e.g., "shoes," "they," "the leather") refer to the same entity. Traditional models often fail to link these mentions, leading to fragmented or incorrect sentiment assignments.

2. Problem Statement: The Coreference Gap

In a typical review, a user might introduce a product feature and then refer back to it using pronouns. Standard models often treat these as separate entities or ignore the sentiment associated with pronouns entirely. This results in an incomplete understanding of the reviewer's intent.

Comparison between traditional ABSC and the proposed coreference-aware NTSA method.

As shown in Figure 1, traditional decoders might identify "leather" as positive and "toe area" as negative but fail to recognize that "they" (referring to the work shoes) carries a neutral or mixed overall sentiment. The proposed method (NTSA) explicitly identifies these coreferential mentions to build a unified entity representation.

3. Methodology

The researchers propose a dual-path framework based on RoBERTa, designed to learn sentiment extraction and coreference resolution simultaneously.

Architecture of the joint multi-task learning framework featuring dual RoBERTa paths and token-level constraints.

3.1 Dual-Path Architecture

The framework consists of two primary components as illustrated in Figure 2:

  1. DTSA Path: Responsible for extracting opinion targets, classifying sentiment (Positive, Negative, Mixed), and identifying hierarchical relationships (e.g., recognizing that "toe area" is a sub-component of "shoes").
  2. Coreference Path: Identifies mentions across the document that refer to the same underlying entity.

3.2 Key Innovations

  • Joint Learning and Parameter Sharing: The two paths share underlying RoBERTa parameters, allowing the model to leverage coreference context for better sentiment understanding.
  • Token-Level Coreference Constraint (TC): This module aligns the representations between the two paths, ensuring that the model's internal representation of an entity remains consistent across its various mentions (e.g., name vs. pronoun).
  • Cluster-Based Metric: To address the limitations of strict string-matching metrics, the authors developed a "loose matching" approach. This metric grants credit if a model correctly identifies sentiment for any member of a coreference cluster, rather than requiring an exact match with a specific gold-standard mention.

4. Experimental Setup and Datasets

The model was evaluated on the DTSA dataset, covering four domains: Books, Clothing, Hotels, and Restaurants. To facilitate training, the authors augmented the dataset with coreference annotations from two sources:

  • Human Experts: Manual clustering of entities.
  • LLM-Assisted (ChatGPT): Using GPT-3.5-Turbo to generate clusters, which were found to be 92% accurate upon manual verification. This suggests that LLMs can significantly reduce the cost of complex NLP data annotation.

5. Results and Discussion

The proposed joint model achieved state-of-the-art performance with an average F1 score of 47.15%, outperforming the previous best-performing baseline (Seq2Struct*).

A significant finding was the impact of the evaluation metric. When moving from a strict metric to the new cluster-based metric, the model's score improved from 47.15% to 48.94%. This 1.8-point jump suggests that previous benchmarks have systematically underestimated model accuracy by penalizing valid coreferential resolutions. The improvements were most notable in the Book and Clothing domains, which typically feature more complex, multi-sentence descriptions.

6. Conclusion

This research demonstrates that explicitly modeling coreference is vital for document-level sentiment analysis. By integrating coreference resolution into a joint learning framework and adopting more flexible evaluation metrics, the researchers have provided a more robust and realistic approach to understanding complex consumer reviews. Future work will focus on reducing noise in specific domains and optimizing LLM-based annotation pipelines.

Summary updated

BibTeX
@inproceedings{cai2024joint,
  author = {Hongjie Cai and Heqing Ma and Jianfei Yu and Rui Xia},
  title = {{A Joint Coreference-Aware Approach to Document-Level Target Sentiment Analysis}},
  booktitle = {Annual Meeting of the Association for Computational Linguistics},
  year = {2024},
  doi = {10.18653/v1/2024.acl-long.657},
}
Figure