Graph Ensemble Learning over Multiple Dependency Trees for Aspect-level Sentiment Classification

Xiaochen Hou, Peng Qi, Guangtao Wang, Rex Ying, Jing Huang, Xiaodong He, Bowen Zhou

Introduction

Aspect-level Sentiment Classification (ALSC) aims to identify the sentiment polarity (positive, negative, or neutral) toward specific aspect terms within a sentence. Recent advancements in the field have leveraged dependency-based Graph Neural Networks (GNNs) to capture long-range syntactic dependencies between aspect terms and their corresponding opinion words. However, these models are often "brittle," as their performance is heavily dependent on the accuracy of the underlying automated syntax parser. If a parser fails to correctly identify a relationship—a common occurrence in informal text like social media—the downstream sentiment model typically fails as well.

The GraphMerge Framework

To address this vulnerability, this research introduces GraphMerge, a novel ensemble technique that shifts the combination of multiple models from the output level to the graph level. Instead of relying on a single, potentially flawed dependency tree, GraphMerge constructs a unified "ensemble graph" by taking the union of edges from multiple distinct parsers.

The GraphMerge architecture, showing the integration of BERT features with a graph ensemble processed by an RGAT.

Methodology

  1. Graph Construction: The framework utilizes three different parsers: Stanford CoreNLP, Stanza, and the Berkeley neural parser. By taking the union of all predicted edges, the model creates a dense graph where a connection exists if any single parser identifies it.
  2. Relational Graph Attention Network (RGAT): To process the resulting multi-edged graph, the authors employ an RGAT. This allows the model to distinguish between different edge types (parent-to-child, child-to-parent, and self-loops), preserving the directional intent of the original syntax.
  3. Feature Representation: Word representations are generated using a pre-trained BERT base, combined with positional encodings to maintain awareness of word order within the sentence.

Key Insights: The Power of the Union

A common concern with merging multiple trees is the introduction of noise. However, the authors found that the union of edges actually benefits the model by reducing the graph diameter. This shorter path between aspects and opinion words facilitates easier learning for the GNN without requiring excessive layers that might lead to over-smoothing.

Examples of dependency edges identified by different parsers, highlighting how the union captures a broader range of potential syntactic relations.

In ablation studies, an "Edge Intersection" strategy (keeping only edges agreed upon by all parsers) performed significantly worse than the "Union" strategy. The union provides a comprehensive set of syntactic hypotheses, and the GNN’s attention mechanism naturally learns to prioritize the most reliable connections while ignoring parser-induced noise.

Results and Robustness

GraphMerge achieved state-of-the-art (SOTA) results across several benchmark datasets:

  • SemEval 2014: Reached 87.32% accuracy on Restaurant reviews and 81.35% on Laptop reviews.
  • Twitter Dataset: Showed strong performance gains, proving the ensemble's ability to mitigate parser errors on informal, ungrammatical text.
  • Aspect Robustness (ARTS): The model demonstrated superior resilience to adversarial changes in sentence structure compared to BERT-only or single-tree models.

Conclusion

GraphMerge offers a computationally efficient alternative to traditional ensemble methods. By merging graphs rather than averaging model outputs, it achieves ensemble-level robustness with the parameter count of a single-tree model. This approach effectively bridges the gap between syntactic theory and the practical realities of noisy, real-world data.

Summary updated

BibTeX
@inproceedings{hou2021graph,
  author = {Xiaochen Hou and Peng Qi and Guangtao Wang and Rex Ying and Jing Huang and Xiaodong He and Bowen Zhou},
  title = {{Graph Ensemble Learning over Multiple Dependency Trees for Aspect-level Sentiment Classification}},
  booktitle = {Conference of the North American Chapter of the Association for Computational Linguistics},
  year = {2021},
  doi = {10.18653/v1/2021.naacl-main.229},
}
Figure