Introduction
Aspect-Level Sentiment Classification (ALSC) is a critical task in natural language processing that involves identifying the sentiment expressed toward specific entities or aspects within a sentence. For example, in the sentence "The battery is great, but the software is buggy," a model must distinguish between the positive sentiment for "battery" and the negative sentiment for "software."
While BERT-based models have become the standard for ALSC, they frequently encounter the "data sparsity" problem. Models often struggle to generalize when an aspect has very few training examples (the "long-tail" problem). This paper introduces AR-BERT (Aspect-Relation BERT), which addresses this by integrating external relational data from Knowledge Graphs (KGs) like DBpedia. By leveraging the fact that KGs already contain relationships between entities (e.g., "Windows 8" is related to "Windows 7"), the model can "borrow" sentiment signals from better-represented neighbors to improve predictions on rare aspects.
Methodology
Integrating massive KGs into deep learning models presents two primary challenges: scalability and noise. AR-BERT addresses these through three core innovations:
1. Two-Level Global-Local Embedding
To handle the millions of nodes in DBpedia, the authors employ a hierarchical embedding scheme. First, the entire KG is partitioned into a "cluster graph" to capture global structure. Second, a specific "subgraph" of entities relevant to the target dataset is extracted to capture local context. These structures are processed using GraphSAGE to generate compact, informative entity embeddings.
2. Incorrect Disambiguation Detection (IDD)
Automated "Wikifiers" used to link text to KG entities are often noisy. To prevent incorrect links from corrupting the model, AR-BERT uses a "probing function." This function compares the similarity of aspects in the BERT embedding space with their similarity in the KG space. If the contexts are too contradictory, the model "zeros out" the graph contribution.
3. Multi-modal Explanations
The framework includes a post-hoc explanation model that identifies whether the text, the graph, or both modalities were most significant for a specific prediction, providing transparency into the model's decision-making process.
Experimental Setup
The model was evaluated on three standard ALSC benchmarks: LAPTOP (SemEval 2014), REST (Restaurant reviews), and TWITTER. The knowledge source utilized was the DBpedia page links graph, containing approximately 22 million nodes and 173 million edges.
Results and Discussion
AR-BERT consistently outperformed state-of-the-art baselines, such as BERT-ADA and SDGCN-BERT, by 2.5 to 4.1 percentage points in Macro-F1 scores.
Solving Data Scarcity
The most significant performance gains were observed for "long-tail" aspects with fewer than 20 training examples. By leveraging KG neighbors, the model effectively bypassed the lack of direct training data, demonstrating the power of external knowledge transfer.
Robustness to Noise
The IDD technique proved highly effective, correctly identifying over 80% of incorrect entity disambiguations across the datasets. This ensures that the integration of external knowledge remains a net positive, even when the underlying linking tools are imperfect.
Conclusion
AR-BERT demonstrates that integrating Knowledge Graphs into BERT-based models can significantly mitigate data sparsity in ALSC. While the model currently relies on static KG snapshots and specific fusion techniques, future work will explore dynamic graph updates and more sophisticated cross-attention mechanisms to further bridge the gap between text and structured knowledge.
Summary updated