Abstract
This study introduces a novel computational linguistics framework to quantify the intensity of complaints on social media. While traditional sentiment analysis distinguishes between positive and negative emotions, it often fails to capture the nuanced spectrum of grievance severity. By developing a dataset of over 3,000 Chinese Weibo posts and utilizing transformer-based models, this research demonstrates that complaint intensity is a distinct metric from general sentiment and serves as a critical indicator for predicting topic virality.
Introduction
In the landscape of social media monitoring, identifying a complaint is only the first step. There is a significant difference between a minor grievance, such as "the coffee was lukewarm," and a severe accusation like "this company is a fraudulent scam." These represent vastly different levels of reputational risk. Prior research has categorized complaints into types (e.g., accusation, disapproval) but has largely ignored the intensity within those categories. This paper addresses this gap, arguing that measuring intensity is essential for organizations to prioritize feedback and understand the "halo effect"—where a single negative experience triggers a broader critique of a brand.
Methodology
The researchers employed three primary strategies to analyze complaint intensity:
- Annotation via Best-Worst Scaling (BWS): To avoid the inconsistencies of traditional 1-5 Likert scales, the study used BWS. Annotators compared sets of four posts to identify the least and most intense complaints, resulting in stable intensity scores ranging from -1 to 1.
- Predictive Modeling: The study benchmarked several architectures, including Support Vector Regression (SVR), Bidirectional LSTMs, and pre-trained transformers such as BERT, RoBERTa, and ERNIE.
- Cross-Lingual Analysis: The authors compared Chinese Weibo posts with English Twitter datasets to identify cultural differences in how grievances are expressed, focusing on directness and the use of irony.
Linguistic Insights: Complaints vs. Sentiment
A key finding is that complaint intensity is not synonymous with negative sentiment. While they correlate, the underlying linguistic features differ. Sentiment models typically focus on emotional keywords, whereas complaint models prioritize the specific reasons for the grievance.
Furthermore, the study identified a "halo effect" in text: stronger complaints are generally longer and more detailed. Highly dissatisfied users tend to expand their critique from a primary issue to peripheral targets, such as attacking support staff or unrelated services.
Results and Impact
The RoBERTa model emerged as the most effective for predicting intensity, achieving a Pearson’s correlation of 0.79. Beyond classification, the practical utility of measuring intensity was demonstrated in popularity prediction. By incorporating "complaint density"—the ratio of high-intensity complaints to total posts—into a baseline model, the researchers significantly improved the accuracy of predicting which topics would go viral.
The inclusion of intensity features reduced the Root Mean Square Error (RMSE) from 0.41 to 0.35, proving that intense early complaints are a reliable early-warning signal for trending topics.
Conclusion
This research establishes complaint intensity as a vital metric for social media analysis. While models still face challenges with irony and implicit dissatisfaction, the ability to quantify the severity of grievances offers profound benefits for crisis management and public sentiment tracking. Future work will focus on expanding the dataset across more diverse domains, such as e-commerce and public utilities, to further refine these predictive capabilities.
Summary updated