Lexicon information in neural sentiment analysis: a multi-task learning approach

Jeremy Barnes, Samia Touileb, Lilja Øvrelid, Erik VelldalNODALIDA 2019 · OpenReview

Abstract

This paper explores the use of multi-task learning (MTL) for incorporating external knowledge in neural models. Specifically, we show how MTL can enable a BiLSTM sentiment classifier to incorporate information from sentiment lexicons. Our MTL set-up is shown to improve model performance (compared to a single-task set-up) on both English and Norwegian sentence-level sentiment datasets. The paper also introduces a new sentiment lexicon for Norwegian.

Introduction

Modern sentiment analysis typically relies on neural networks like BiLSTMs, which excel at capturing context but often function as "black boxes" requiring extensive training data. Conversely, traditional sentiment lexicons—curated lists of words with assigned polarities—offer transparency and handle rare words effectively. This research proposes a bridge between these paradigms using Multi-Task Learning (MTL) to integrate the "distilled wisdom" of lexicons into neural classifiers without making the model overly rigid.

Methodology

The authors implement an MTL architecture utilizing hard parameter sharing to infuse lexicon-based knowledge into the neural representation.

The proposed multi-task learning architecture featuring shared lower layers for lexicon prediction (auxiliary task) and sentiment classification (main task).

The model consists of three primary components:

  1. Shared Lower Layers: An initial embedding layer followed by a fully connected layer. These layers are shared across both tasks to learn a unified representation.
  2. Auxiliary Task (Lexicon Prediction): This head is trained to predict the polarity of individual words based on an external lexicon. By using word embeddings as input, the model learns to generalize sentiment to words not explicitly present in the lexicon.
  3. Main Task (Sentence Classification): The "sentiment-infused" representations are passed to a BiLSTM with max-pooling and a softmax layer to determine the overall sentiment of the sentence.

Datasets and Resources

The approach was evaluated on two languages:

  • English: Utilizing the Stanford Sentiment Treebank (SST) and the Hu and Liu (2004) lexicon.
  • Norwegian: Utilizing the NoReCeval dataset.
  • Norwegian Lexicon: A significant contribution of this work is the creation of a new Norwegian sentiment lexicon. This was developed by machine-translating the Hu and Liu lexicon, followed by manual curation and expansion using the SCARRIE full-form lexicon to include over 20,000 inflected forms.

Results

The MTL framework consistently outperformed Single-Task Learning (STL) and traditional baselines across both languages.

Performance comparison (Macro F1) of various models on the SST and NoReCeval datasets, highlighting the gains achieved by the MTL approach.

As shown in the results table, the MTL model achieved a Macro F1 of 42.4 on the English SST dataset, a notable improvement over the STL baseline of 37.8. Similarly, on the Norwegian NoReCeval dataset, the MTL model reached an F1 of 52.8, surpassing the STL performance of 51.2.

Discussion and Insights

The auxiliary lexicon prediction task serves as an effective regularizer, preventing the model from overfitting to specific sentence-level patterns by enforcing a ground-truth understanding of word-level polarity. This allows the model to generalize sentiment to semantically similar words (e.g., learning "superb" is positive if "excellent" is in the lexicon). Furthermore, error analysis indicates that lexicon information particularly aids in distinguishing subtle sentiment in neutral sentences and minority classes.

Limitations and Future Work

While successful, the current model has limitations. The lack of subword analysis may hinder the detection of sentiment in morphologically complex words, and the word-level focus struggles with multi-word expressions and idioms. Future work could extend the auxiliary task from binary polarity to sentiment intensity or specific emotion labels to provide a richer training signal.

Summary updated

Abstract

This paper explores the use of multi-task learning (MTL) for incorporating external knowledge in neural models. Specifically, we show how MTL can enable a BiLSTM sentiment classifier to incorporate information from sentiment lexicons. Our MTL set-up is shown to improve model performance (compared to a single-task set-up) on both English and Norwegian sentence-level sentiment datasets. The paper also introduces a new sentiment lexicon for Norwegian.
BibTeX
@inproceedings{barnes2019lexicon,
  author = {Jeremy Barnes and Samia Touileb and Lilja Øvrelid and Erik Velldal},
  title = {{Lexicon information in neural sentiment analysis: a multi-task learning approach}},
  booktitle = {NODALIDA},
  year = {2019},
  url = {https://openreview.net/forum?id=suphkMAQP9o},
}
Figure