Abstract
Sentiment analysis on social media faces significant hurdles due to informal language and the high cost of manual data labeling. This paper presents Constrained Symmetric Nonnegative Matrix Factorization (CSNMF), an automated method for generating high-coverage sentiment dictionaries. By integrating linguistic knowledge from WordNet with real-world usage patterns from Digg.com, the method produces a lexicon that outperforms traditional dictionaries. Experimental results demonstrate that this hybrid approach achieves superior precision and recall, effectively bridging the "informality gap" in social media sentiment analysis.
The Challenge of Social Media Sentiment
Traditional sentiment analysis tools often rely on lexicons like the General Inquirer (GI), which are designed for formal text. Social media, however, is characterized by slang, abbreviations, and evolving terminology (e.g., "diggable"). Machine learning models can address this but require massive, human-labeled datasets that are expensive to maintain. Lexicon-based approaches are more scalable but suffer from low coverage of informal terms. Furthermore, human agreement on the sentiment of social media comments is remarkably low—often around 44%—making manual labeling inherently difficult.
Methodology: The CSNMF Pipeline
The authors propose a multi-step pipeline to automatically expand a small seed of 52 known adjectives into a comprehensive lexicon:
- Seed Expansion: Initial positive and negative words are expanded using synonyms and antonyms from WordNet.
- Corpus Expansion: The system crawls a large social media corpus (Digg.com) to identify adjectives linked by conjunctions. Following linguistic principles, "and" typically links words of similar polarity, while "but" links opposites.
- CSNMF Algorithm: The core of the method is a semi-supervised clustering algorithm. It uses matrix factorization to group words into positive and negative clusters based on "attraction" (synonyms, "and" links) and "repulsion" (antonyms, "but" links). Crucially, the algorithm assigns a numerical sentiment strength score to each word, reflecting the confidence of its polarity assignment.
Experimental Setup and Datasets
The study utilized a 2GB corpus from Digg.com, containing over 1.8 million comments. To establish a ground truth for social media language, the authors used Amazon Mechanical Turk (AMT) to collect human labels for over 7,000 adjectives and 8,000 comments. This "social-media-specific" ground truth allowed for a more accurate evaluation of the generated dictionary compared to using formal benchmarks alone.
Key Results and Insights
The CSNMF method demonstrated significant improvements over standard clustering techniques and traditional lexicons.
- Precision and Confidence: A key finding was the correlation between the algorithm's sentiment score and its accuracy. As shown in the performance evaluation, words with higher sentiment scores (representing higher confidence) yielded significantly better precision when compared against human labels.
- Superior Coverage: The generated dictionary achieved a recall of 89.41%, catching far more sentiment-carrying words than the human-labeled AMT dictionary or the General Inquirer. In real-world classification tasks, this increased breadth proved more valuable than the "perfect" but limited labels of smaller manual dictionaries.
- Hybrid Knowledge: The results confirm that combining "top-down" linguistic relationships (WordNet) with "bottom-up" usage data (Digg) is more effective than using either source in isolation.
Conclusion and Future Directions
The CSNMF approach provides a robust, automated way to keep sentiment dictionaries up-to-date with the rapidly changing language of social media. The effectiveness of this method is visualized in applications like "Social Sense Diffusion," which tracks opinion shifts over time.
While the current focus on adjectives provides high precision, future work could expand the model to include nouns and verbs, and better address the complexities of sarcasm and context-dependent polarity.
Summary updated