Introduction
Aspect-Based Sentiment Analysis (ABSA) aims to identify sentiment polarities towards specific entities or aspects within a sentence. A significant challenge in ABSA is the domain-dependent nature of sentiment. For example, the word "hot" typically conveys a negative sentiment in the context of laptop hardware (indicating overheating) but a positive sentiment in a restaurant review (indicating fresh, warm food).
Training robust ABSA models requires diverse, multi-domain datasets. However, such data is often siloed across different organizations or devices, restricted by privacy regulations. While Federated Learning (FL) offers a decentralized training paradigm, standard FL approaches struggle with the data heterogeneity inherent in NLP tasks. They often produce a "one-size-fits-all" model that fails to capture the nuanced, domain-specific sentiment patterns required for accurate ABSA.
Proposed Method: TM-FL
To address these challenges, we introduce Topic Memory Federated Learning (TM-FL). This framework integrates a Topic Memory (TM) module into the federated architecture to serve as a privacy-preserving proxy for domain context.
Federated Architecture and Privacy
TM-FL utilizes a server-node architecture designed for privacy. Instead of sharing raw text, nodes transform local data into hidden vector representations. These encrypted vectors are transmitted to a central server, which performs the forward pass, calculates loss, and returns gradients to update the local models. This ensures that sensitive raw data never leaves the local environment.
Topic Memory Module
The core innovation is the Topic Memory module, which operates alongside a BERT encoder. The process involves:
- Neural Topic Modeling: A Variational Auto-Encoder (VAE) is pre-trained on a large, unlabeled corpus to learn a latent topic-vocabulary matrix.
- Topic Mapping: When the server receives an encrypted input, the TM module maps the constituent words to topic embeddings.
- Attention Mechanism: An attention mechanism identifies which latent topics are most relevant to the current input sentence.
- Integration: The module generates a categorical topic vector that is concatenated with the BERT-derived sentence representation. This combined feature set allows the classifier to interpret sentiment cues through the lens of the identified topic (e.g., "electronics" vs. "food").
Experimental Evaluation
We evaluated TM-FL on three benchmark ABSA datasets: LAP14 (laptops), REST14 (restaurants), and TWITTER (social media). The underlying topic model was trained on approximately 300,000 sentences from diverse sources including Yelp, Amazon, and IMDb.
Key Results
TM-FL consistently outperformed standard FL baselines across all metrics. On the LAP14 dataset, TM-FL achieved an accuracy of 82.29%, surpassing the 81.35% achieved by standard FL. Similar gains were observed in Macro-F1 scores, particularly in domains with highly specialized vocabularies.
Qualitative Analysis
As shown in the case studies above, TM-FL demonstrates a superior ability to handle contradictory sentiment cues. In Sentence (1) from the LAP14 dataset, the baseline model fails to recognize that "hot" is negative for a computer. Conversely, in Sentence (2) from REST14, TM-FL correctly identifies "hot" as positive for food. This confirms that the Topic Memory module effectively recovers the contextual information typically lost in isolated data silos, performing nearly as well as models trained on pooled, non-private data.
Conclusion
TM-FL provides a robust solution for multi-domain ABSA in privacy-sensitive environments. By using latent topics as a proxy for domain context, the model bridges the "isolation gap" in federated learning without compromising data security. Future work will focus on scaling this approach to thousands of nodes and exploring online topic models that can adapt dynamically as new data sources join the network.
Summary updated