Deep Mask Memory Network with Semantic Dependency and Context Moment for Aspect Level Sentiment Classification

Peiqin Lin, Meng Yang, Jianhuang LaiIJCAI 2019 · DOI

TL;DR

This paper introduces DMMN-SDCM, a novel deep learning framework for aspect-level sentiment classification. It enhances traditional memory networks by incorporating semantic dependency information to guide attention and modeling inter-aspect relationships, along with a context moment learning task to capture overall sentence sentiment, achieving state-of-the-art results.

What Problem Does This Paper Address?

Aspect-level sentiment classification (ABSC) aims to determine the sentiment (positive, negative, neutral) towards specific aspects within a sentence. Unlike sentence-level sentiment analysis, ABSC requires a more granular understanding, as different aspects in the same sentence can carry opposing sentiments (e.g., "The food was great, but the service was terrible").

Previous deep learning approaches, while effective, often relied on simple word location information to guide attention mechanisms. This overlooked crucial semantic relationships between words and aspects, and also failed to fully leverage the interplay between multiple aspects within a sentence. The authors identify these limitations: the inability of traditional methods to capture important semantic clues and the underutilization of global sentence sentiment and inter-aspect relationships.

Methods

The proposed framework, DMMN-SDCM (Deep Mask Memory Network with Semantic Dependency and Context Moment), builds upon deep memory networks. Its key innovations are:

  • Semantic-Dependency-Mask Attention: Instead of relying solely on word positions, this module uses semantic parsing (dependency trees) to mask the context memory. This allows the attention mechanism to dynamically select relevant parts of the memory for different computational layers, guided by the semantic relationships between words and the target aspect.
  • Inter-Aspect Semantic Modeling: This module explicitly models the relationships between different aspects within a sentence. It integrates semantic dependency information and attention mechanisms to capture how other aspects influence the sentiment of the target aspect.
  • Context-Moment Sentiment Learning: This novel auxiliary task aims to learn the overall sentiment distribution of the entire sentence. By capturing the first (mean) and second (variance) moments of this distribution, the model gains a background understanding of the sentence's sentiment, which can inform the classification of individual aspects. This is particularly useful for understanding coordination or comparison between aspects.

The overall architecture consists of an embedding module, memory building module (using BiLSTM), the proposed semantic-dependency-mask attention, inter-aspect semantic modeling, context-moment sentiment learning, and a final classification module.

Datasets

The experiments were conducted on the SemEval 2014 Task 4 datasets, specifically the Laptop and Restaurant review datasets. These datasets contain aspect-based sentiment annotations. The authors removed the "conflict" category due to its small size, which would have made the dataset highly unbalanced.

Key Insights and Ideas

  • Semantic Structure Over Location: The paper convincingly demonstrates that leveraging semantic parsing (dependency trees) provides a richer and more accurate way to guide attention than simple word positional information. This allows the model to focus on semantically relevant context words, even if they are not adjacent to the aspect.
  • Inter-Aspect Synergy: Explicitly modeling the relationships between aspects within a sentence significantly improves performance. This highlights that aspects don't exist in isolation; their co-occurrence and relationships provide valuable context.
  • Global Sentiment as Context: The context-moment learning task is a clever way to inject global sentence sentiment into the aspect-level classification. Understanding whether a sentence is generally positive or negative, or how aspects relate to each other (e.g., comparison vs. coordination), provides a crucial backdrop for individual aspect sentiment.
  • Moments as Sentiment Descriptors: Using the mean and variance of aspect sentiments as descriptors for the overall sentence sentiment is an intuitive and effective approach. The mean captures the general sentiment, while the variance indicates the degree of agreement or disagreement among aspects.

Key Results

The DMMN-SDCM model achieved state-of-the-art performance on the SemEval 2014 datasets for both Laptop and Restaurant reviews.

  • On the Laptop dataset, the model achieved an accuracy of 77.59% and a Macro-F1 score of 73.61%.
  • On the Restaurant dataset, the model achieved an accuracy of 81.16% and a Macro-F1 score of 71.50%.

Ablation studies showed that each of the proposed modules (semantic dependency mask attention, inter-aspect modeling, and context-moment learning) contributed positively to the overall performance. Specifically, integrating semantic dependency information alone outperformed previous methods like RAM, and further improvements were seen with the addition of inter-aspect modeling and context-moment learning.

Limitations and Future Work

  • Limitations: The paper notes that the "conflict" sentiment category was removed from the SemEval datasets due to its small size, which might limit the model's performance on such nuanced cases. The reliance on external semantic parsers (like spaCy) introduces a dependency on the quality and availability of these tools.
  • Future Work: The authors suggest that the context moment learning could be extended to higher-order moments (like skewness) to capture more complex sentiment distributions. Further exploration of different semantic parsing techniques and their integration into memory networks could also be beneficial. The potential for applying this framework to other NLP tasks that involve understanding relationships between entities and their sentiments is also implied.

Summary updated

BibTeX
@inproceedings{lin2019deep,
  author = {Peiqin Lin and Meng Yang and Jianhuang Lai},
  title = {{Deep Mask Memory Network with Semantic Dependency and Context Moment for Aspect Level Sentiment Classification}},
  booktitle = {International Joint Conference on Artificial Intelligence},
  year = {2019},
  doi = {10.24963/ijcai.2019/707},
}
Figure