Deep Multi-Task Learning for Aspect Term Extraction with Memory Interaction

Xin Li, Wai LamEMNLP 2017 · OpenReview

Abstract

Aspect Term Extraction (ATE) is a fundamental task in Aspect-Based Sentiment Analysis (ABSA). Traditional methods often struggle with informal text and ignore the co-dependency between aspect terms and opinion words. This paper introduces the Memory Interaction Network (MIN), a deep multi-task learning framework that jointly extracts aspects and opinions. By utilizing a shared memory mechanism for interaction between specialized LSTMs and incorporating a sentence-level sentiment constraint, MIN achieves state-of-the-art performance on benchmark datasets without relying on error-prone dependency parsers.

Introduction

In Aspect-Based Sentiment Analysis, identifying specific features of a product or service—known as Aspect Term Extraction—is a critical first step. However, existing approaches face two major hurdles. First, they often ignore the "opinion clue," failing to leverage the strong co-dependency between aspect terms (e.g., "crust") and opinion words (e.g., "soggy"). Second, models that do attempt to link these elements frequently rely on dependency parsers, which are notoriously brittle when applied to the informal grammar and "text-speak" common in online reviews.

To address these issues, we propose the Memory Interaction Network (MIN), a purely neural approach that captures the functional relationship between aspects and opinions while remaining robust to messy, real-world text.

Methodology

The core of the MIN framework consists of three LSTMs working in tandem to perform multi-task learning:

  • A-LSTM (Aspect Task): Specifically designed to identify aspect terms.
  • O-LSTM (Opinion Task): Designed to identify opinion words. To overcome the lack of "gold standard" labels for opinion words in most datasets, we utilize sentiment lexicons and high-precision rules to generate pseudo-labels for training.
  • S-LSTM (Sentiment Constraint): A vanilla LSTM that determines if a sentence is sentimental. This acts as a filter to prevent the model from hallucinating aspects in neutral, factual sentences.

The Memory Interaction Mechanism

The A-LSTM and O-LSTM communicate through a shared memory mechanism involving three neural operations:

  1. READ: Each LSTM accesses a small window of recent hidden states (the "memory").
  2. DIGEST: The model calculates an "indicator score" for these memories. For the A-LSTM, it weights memories based on their likelihood of being opinion words and their physical proximity to the current word.
  3. INTERACT: The digested summary from the opinion task is fused into the update gates of the aspect task (and vice versa), allowing the model to adjust its predictions based on nearby opinionated context.

Experimental Results

We evaluated MIN on two widely used benchmarks from the SemEval ABSA challenges: the Laptop domain (D1) and the Restaurant domain (D2). MIN outperformed several strong baselines, including traditional CRF-based methods and the previous state-of-the-art recursive neural network (RNCRF).

Comparison of MIN against baseline models on D1 and D2 datasets.

As shown in the results, MIN achieved an F1 score of 77.58% on the Laptop dataset and 73.44% on the Restaurant dataset, establishing a new state-of-the-art for these benchmarks.

Ablation Study

To understand the contribution of each component, we conducted an ablation study. The results confirm that both the interaction between tasks and the sentiment constraint are vital for performance.

Ablation study of the MIN framework components showing the impact of removing specific modules.

Removing the O-LSTM (Interaction) caused the F1 score to drop by approximately 1.6%, proving that "talking" to the opinion extractor significantly aids aspect detection. Similarly, removing the S-LSTM (Sentiment Constraint) caused a drop of ~1.5%, confirming that sentence-level context is essential for filtering noise in neutral contexts.

Conclusion

The Memory Interaction Network demonstrates that neural temporal memory windows can effectively replace structural dependency trees for ATE. By treating aspect extraction as a multi-task problem with sentiment constraints, MIN provides a robust solution for processing informal user reviews. Future work may extend this framework to jointly predict aspect categories or sentiment polarity in a single pass.

Summary updated

BibTeX
@inproceedings{li2017deep,
  author = {Xin Li and Wai Lam},
  title = {{Deep Multi-Task Learning for Aspect Term Extraction with Memory Interaction}},
  booktitle = {Conference on Empirical Methods in Natural Language Processing},
  year = {2017},
  url = {https://openreview.net/forum?id=SkVE8zz_ZS},
}
Figure