Introduction
Word-Level Contextual Sentiment Analysis (WCSA) is the task of determining the sentiment of a specific word within its context—for example, recognizing that "good" in the phrase "not good" is actually negative. In industrial applications, two things are usually required: interpretability (knowing why a model made a choice) and practicality (not needing millions of hand-labeled examples).
Existing methods usually fail at one of these. Rule-based systems are interpretable but brittle, while deep learning models often act as "black boxes," making it impossible to pinpoint exactly which contextual factor flipped a word's sentiment. This paper introduces SINN (Sentiment Interpretable Neural Network), a novel architecture designed to perform word-level sentiment analysis while remaining fully transparent.
Methodology
The researchers introduced two core innovations: the SINN architecture and the LEXIL training strategy.
The SINN Architecture
SINN is structured into four distinct, human-readable layers that decompose sentiment into its constituent parts:
- Word-level Original Sentiment Layer (WOSL): Stores the "base" sentiment of a word (e.g., "excellent" is inherently +1.0).
- Local Word-level Context Layer (LWCL): Uses LSTMs to detect if a word’s sentiment is shifted by its immediate neighbors (e.g., "not," "never").
- Global Word-level Context Layer (GWCL): Uses an attention mechanism to determine how much a specific word matters to the overall sentiment of the entire review.
- Word-level Contextual Sentiment Layer (WCSL): The final output, calculated by multiplying the values of the previous three layers.
Lexical Initialization Learning (LEXIL)
To force the model to stay interpretable, the authors use LEXIL. They initialize the WOSL layer using a very small sentiment dictionary (just a few hundred words). This "seed" knowledge guides the rest of the network to learn local and global contexts that align with how humans perceive sentiment shifts, allowing the model to generalize to thousands of other words through propagation.
Key Results
The authors tested SINN across different languages and domains, including Japanese economic trends (EcoRev) and English tweets (Sentiment 140).
- Interpretability: In evaluations of WOSL (original sentiment) and LWCL (local shifts), SINN significantly outperformed baselines like PMI and NegRNN, often achieving Macro F1 scores 10–20 points higher.
- WCSA Ability: On the core task of assigning contextual sentiment, SINN (with 100-200 seed words) outperformed state-of-the-art models like GINN across almost all datasets.
- Qualitative Success: Visualizations confirmed the model's ability to correctly identify sentiment shifts, such as recognizing that "great" became negative when preceded by "not" in English, and similarly handling complex negations in Japanese economic contexts.
Conclusion
By defining contextual sentiment as a product of base sentiment, local shift, and global importance, SINN makes the model's "thought process" mathematically transparent. The approach is language-agnostic and practical, requiring only a small initial lexicon to achieve high performance. Future work will explore complex sarcasm and expansion into specialized fields like legal or medical documentation.
Summary updated