A Non-negative Matrix Tri-factorization Approach to Sentiment Classification with Lexical Prior Knowledge

Tao Li, Yi Zhang, Vikas SindhwaniACL 2009 · OpenReview

Abstract

Tao Li, Yi Zhang, Vikas Sindhwani. Proceedings of the Joint Conference of the 47th Annual Meeting of the ACL and the 4th International Joint Conference on Natural Language Processing of the AFNLP.

2009.

Introduction

Sentiment classification typically relies on either large-scale manually labeled datasets (supervised learning) or pre-defined sentiment lexicons (dictionary-based methods). However, both approaches have significant drawbacks: supervised data is costly to acquire, and general-purpose dictionaries often fail to capture domain-specific nuances. For instance, the word "small" might be positive for a mobile device but negative for a hotel room. This paper proposes a framework that bridges these methodologies by utilizing Non-negative Matrix Tri-factorization (NMTF) to incorporate both general lexical knowledge and domain-specific unlabeled data with minimal manual supervision.

Methodology: Non-negative Matrix Tri-factorization

The proposed approach, Matrix Factorization with Lexical Knowledge (MFLK), represents the corpus as a term-document matrix $X$. This matrix is decomposed into three components: $X \approx FSG^T$, where $F$ represents word-class associations (sentiment of words), $G$ represents document-class associations (sentiment of documents), and $S$ captures the interaction between word and document clusters.

To guide the learning process, the authors introduce two primary constraints into the optimization objective:

  1. Lexical Prior: The word matrix $F$ is penalized if it deviates significantly from a known sentiment lexicon.
  2. Document Labels: The document matrix $G$ is constrained to align with the small set of manually labeled examples provided.

This "dual-supervision" allows information to propagate bidirectionally: labeled documents help identify new sentiment-bearing words, while known sentiment words assist in classifying unlabeled documents.

Experimental Results

The framework was evaluated across four datasets: Movie Reviews (IMDB), Lotus Blogs, Political Blogs, and Amazon Reviews. The authors compared their method against several baselines, including K-Means, Information-Theoretic Co-Clustering (ITCC), and standard supervised models like SVMs.

Performance and Generality

The MFLK model consistently outperformed unsupervised and co-clustering baselines across all test domains. By leveraging the interaction matrix $S$, the model effectively adapts to domain-specific contexts, identifying "polarity switching" where word meanings shift based on co-occurrence patterns in the unlabeled data.

Accuracy comparison of MFLK against baseline methods across four datasets.

Semi-Supervised Robustness

In scenarios with very limited labeled data, the semi-supervised variant (SSMFLK) showed remarkable resilience. When only a small fraction of documents were labeled, SSMFLK outperformed traditional supervised methods like Support Vector Machines (SVM) and Naive Bayes, as well as other semi-supervised graph-based methods.

Performance of SSMFLK compared to supervised and semi-supervised baselines with varying amounts of labeled data.

The study also found that accuracy improves steadily as the size of the sentiment lexicon increases, confirming the model's ability to harvest and utilize prior knowledge effectively. Furthermore, the method remained stable across varying vocabulary sizes, indicating robustness against feature selection noise.

Discussion and Future Work

While the NMTF approach is powerful, it currently operates on a "bag-of-words" assumption, which limits its ability to handle complex linguistic structures like negation or sarcasm. Future research could explore incorporating richer linguistic features, such as dependency graphs or cross-domain transfer learning, to further enhance the model's adaptability and precision.

Summary updated

BibTeX
@inproceedings{li2009non,
  author = {Tao Li and Yi Zhang and Vikas Sindhwani},
  title = {{A Non-negative Matrix Tri-factorization Approach to Sentiment Classification with Lexical Prior Knowledge}},
  booktitle = {Annual Meeting of the Association for Computational Linguistics},
  year = {2009},
  url = {https://openreview.net/forum?id=SyWDf2xdWB},
}
Figure