TL;DR
This paper tackles the challenge of automatically classifying moral foundations in political tweets, which are often short and ambiguous. They propose a novel approach that combines language analysis with behavioral data (retweets and follower networks) using Probabilistic Soft Logic (PSL) to build more accurate and robust models. This work offers a new way to understand how politicians frame issues and how these frames evolve over time.
What Problem Does This Paper Address?
Political discourse on social media, particularly Twitter, is a crucial but challenging area to study. The brevity of tweets often strips away context, making it difficult to understand the underlying moral framing. This ambiguity is exemplified by phrases like "thoughts and prayers," which can carry different moral implications depending on the speaker and context. Traditional methods for analyzing moral foundations, like keyword dictionaries, struggle with the dynamic and often subtle nature of social media language. The authors aim to overcome this by developing computational models that can accurately identify the moral foundations (e.g., Care/Harm, Fairness/Cheating) used by politicians in their tweets, even with limited human annotation.
Methods
The core of this paper's contribution lies in its novel modeling approach. Instead of relying solely on textual content, they integrate behavioral aspects of politicians on Twitter. They use Probabilistic Soft Logic (PSL), a declarative modeling language that allows for the specification of weighted, first-order logic formulas. These formulas are compiled into a graphical model that defines a probability distribution over possible assignments of moral foundations.
The authors build increasingly complex PSL models:
- BASELINE: This model uses only language-based features, including unigrams from the Moral Foundations Dictionary, political slogans, and ideological phrase indicators. This serves as a benchmark.
- +RETWEETS: This model adds retweet information, leveraging the idea that retweets can act as both textual indicators and miniature representations of network structure.
- +FOLLOWING: This model further incorporates the actual social network by including information about who follows whom. The hypothesis is that politicians with similar ideologies (and thus similar moral foundations) tend to follow each other.
- +TEMPORAL: This model introduces temporal information, considering if tweets occur within the same time frame. This is motivated by the observation that discussions around events often peak on the day they occur and then decline.
This approach is described as "weakly-supervised" because it uses high-level rules and can leverage behavioral data to reduce the burden of manual annotation.
Datasets
The authors utilize three datasets for their work:
- Congressional Tweets Dataset: This dataset consists of tweets from the 114th Congress, annotated with moral foundations. It was used for supervised and unsupervised model design and evaluation.
- Senate Tweets 2016: This dataset comprises all available tweets from Senators during 2016, collected via web scraping and the Twitter API. This dataset is made publicly available.
- Congress Tweets: This dataset contains tweets from all congressional members in 2018. For comparison with the Senate Tweets 2016 dataset, only tweets from senators were used.
Key Insights and Ideas
- Behavioral Data is Crucial: Integrating social network information (following) and interaction data (retweets) significantly improves the accuracy of moral foundation classification, outperforming language-only models.
- Temporal Patterns Matter: Considering the timing of tweets can further enhance model performance, suggesting that synchronized discussions around events often involve similar moral framing.
- Weak Supervision is Powerful: The PSL framework, combined with behavioral data, allows for effective classification even in unsupervised settings, potentially reducing the need for extensive manual annotation.
- Moral Framing Evolves: The qualitative analysis reveals distinct patterns in how moral foundations are used by different political parties over time and in response to specific events, highlighting the dynamic nature of political discourse.
Key Results
Quantitative Results:
- Supervised Experiments: The final combined model (+TEMPORAL) achieved an average F1 score of 81.63%, a significant improvement over the BASELINE language-only model (72.49%). The addition of social and behavioral features consistently improved predictions across most moral foundations.
- Unsupervised Experiments: The combined model achieved an average F1 score of 64.75%, a 12.06 point increase over the language-only baseline. This demonstrates the effectiveness of their approach even without explicit supervision.
Qualitative Findings:
- Party-Specific Trends: Republicans showed a stronger emphasis on the "Care" foundation in 2016, which decreased in 2018, with a more even distribution of other foundations. Democrats, on the other hand, used the first four foundations more evenly, with a significant increase in Twitter activity and foundation usage in 2018.
- Event-Specific Framing: The analysis of specific events (e.g., gun violence, women's rights, LGBTQ rights) showed that while "Care" and "Harm" were frequently used by both parties in response to shootings, other foundations like "Fairness" and "Cheating" were employed to discuss justice for victims. The "Authority" foundation saw a decrease in use for Supreme Court nominations over time.
- Transgender Rights: For transgender rights issues, both parties expressed concern using "Care" and "Harm," with a notable emphasis on "Harm" across most groups.
Limitations and Future Work
Limitations:
- The paper acknowledges that the incorporation of retweet information did not always improve scores, potentially due to low quantities of retweet data or retweets being simple copies of original tweets.
- The current study focused on senators; the authors note that the House of Representatives' tweets are not yet included.
Future Work:
- Expand issue coverage to include more trending topics.
- Incorporate tweets from members of the House of Representatives for the last five years.
- Further explore the relationship between political discourse trends and real-world events and policy issues.
Summary updated