TL;DR
This paper introduces a novel framework for automatically identifying the reasons behind stress expressed in social media, moving beyond simply detecting stress itself. By analyzing tweets from politics and airline domains, they demonstrate that word-vector based methods, particularly a cluster vector similarity approach, can effectively pinpoint stress causes, offering valuable insights for targeted interventions.
What Problem Does This Paper Address?
While detecting stress from social media offers a non-intrusive way to monitor mental well-being, understanding why people are stressed is crucial for effective action. Traditional methods like questionnaires or physiological sensors are often intrusive or impractical for large-scale monitoring. Existing social media stress detection primarily focuses on identifying the presence of stress, but lacks the granularity to pinpoint its underlying causes. This gap limits the ability to address the root issues, whether it's understanding customer dissatisfaction with airlines or identifying societal anxieties related to political events. This paper aims to bridge this gap by developing methods to automatically extract stress reasons from tweets.
Methods
The authors propose a multi-stage approach. First, they identify potential "stressors" (reasons for stress) within specific domains (politics and airlines). This involves collecting high-stress tweets, using topic modeling (Latent Dirichlet Allocation with hashtag pooling) to identify clusters of related topics, and then manually refining these clusters to assign descriptive "stressor" titles.
For automatically detecting stress reasons in new tweets, they employ three novel word-vector based methods:
- Maximum Word Similarity: Compares individual words in a tweet to potential stressors and selects the stressor with the highest similarity.
- Context Vector Similarity: Creates a single "context vector" for a tweet by averaging the word vectors of its content words, then compares this to stressor vectors.
- Cluster Vector Similarity: Represents each potential stressor by a "cluster vector" (average of vectors of words in its topic cluster) and compares these to the tweet's context vector.
These methods are then compared against standard machine learning algorithms (SVM, AdaBoost, Logistic Regression) for evaluation.
Datasets
The study utilizes two datasets of public Twitter posts collected via the Tweepy API:
- Politics: Tweets collected using hashtags like "#politics AND #us" and "#uspolitics." An initial set of 22,293 tweets was filtered to 8,163 after removing duplicates and retweets. A subset of 2,205 high-stress tweets was used for stressor identification. For annotation, 2,000 tweets were independently annotated by three human coders.
- Airlines: Tweets collected using hashtags of nine popular airlines. An initial set of 31,457 tweets was filtered to 7,965. A subset of 3,214 high-stress tweets was used for stressor identification. For annotation, 2,000 tweets were randomly selected from the high-stress set and annotated by three human coders.
A new benchmark dataset of tweets annotated with specific stress reasons was created as part of this work.
Key Insights and Ideas
- Moving Beyond "Stress Detection": The core innovation is shifting focus from if someone is stressed to why they are stressed, enabling more actionable insights.
- Domain-Specific Stressors: Stressors are not universal; they are highly dependent on the domain (e.g., "violence" in politics vs. "delay" in airlines).
- Word Vectors as a Powerful Tool: Word vector representations (like Word2Vec and GloVe) are effective for capturing semantic meaning in short, informal texts like tweets, outperforming traditional bag-of-words approaches.
- Context Matters: Aggregating word information into context or cluster vectors (Methods 2 and 3) generally yields better results than looking at individual word similarities, suggesting that the overall sentiment and topic of a tweet are crucial.
- Cluster Vector Superiority: The "cluster vector similarity" method, which leverages pre-defined topic clusters of stressors, emerged as the most accurate approach. This highlights the benefit of incorporating structured knowledge about potential stress causes.
Key Results
- The proposed word-vector based methods, particularly the cluster vector similarity method, achieved higher accuracy in detecting stress reasons compared to standard machine learning algorithms.
- For the Politics dataset, the cluster vector method achieved an accuracy of 63.41%, outperforming SVM (52.48%) and AdaBoost (50.64%).
- For the Airlines dataset, the cluster vector method achieved an accuracy of 67.29%, again outperforming SVM (58.13%) and AdaBoost (54.85%).
- Qualitative analysis revealed that "violence" was the most common stress reason in politics tweets (33.17%), while "delay" was the most common in airline tweets (39.02%).
- High inter-coder agreement (Krippendorff's α values around 70-75%) was achieved for human annotation, indicating a well-defined problem and annotation process.
Limitations and Future Work
- Misleading Hashtags/Content: The methods can be misled by hashtags or words that suggest a different stressor than the actual one (e.g., "#news" leading to "media" instead of "violence").
- Multiple Stressors: Tweets expressing multiple reasons for stress pose a challenge. The current methods assign a single stressor.
- Domain and Source Limitation: The study is limited to two domains (politics and airlines) and one source (Twitter).
Future work suggested includes:
- Expanding the analysis to other domains and social media platforms.
- Automating the process of detecting potential stressors for new domains.
- Developing methods to accommodate and identify multiple stressors within a single tweet.
- Investigating techniques to mitigate the impact of misleading hashtags and content words.
Summary updated