Abstract
Discourse on social media platforms is often plagued by acute polarization, with different camps promoting different perspectives on the issue at hand. While previous research has focused on the affiliation of groups and individuals, we propose a new finer-grained perspective: studying the impartiality of individual messages. We operationalize message impartiality in terms of how discernible the affiliation of its author is and introduce a methodology for quantifying it automatically. Our framework enables us to study the effects of (im)partiality on social media discussions at scale. We show that partial messages are twice as likely to spread than impartial ones, even after controlling for author and topic.
Introduction
Most research on social media polarization focuses on the author—labeling users as "liberal" or "conservative." However, this ignores the nuance of individual utterances. A staunch partisan can still post a neutral news update, and a moderate can post a highly biased "hot take." The challenge is that "impartiality" is deeply subjective and hard for humans to annotate consistently. This paper seeks to operationalize impartiality as a measurable metric and create an automated system that works for emerging events without needing new human labels every time.
Methodology: Affiliation Discernibility
The core of our approach is the concept of Affiliation Discernibility. If a message is truly impartial, an observer should not be able to tell which "side" the author supports. Conversely, if a message contains cues that make the author's affiliation obvious, it is considered partial.
We implement this using the Unigram Method. This technique involves scoring individual words (unigrams) based on how strongly they signal a specific affiliation within a given context. For instance, during the 2013 U.S. government shutdown, terms like "#GOPShutdown" were strong signals for Democrats. A message’s partiality score ($mp\text{-}score$) is determined by its most "discerning" word; if a tweet contains even one highly partisan keyword, the entire message is flagged as partial.
Validation of the Impartiality Metric
To ensure our automated metric reflects human perception, we compared the automated scores against human judgments obtained via Amazon Mechanical Turk. The results showed a strong alignment between the "discernibility" metric and human intuition regarding bias.
As shown in Figure 1, as the level of human agreement on a message's bias increases (the agreement bin), the average partiality rating generated by our model also increases, validating the metric's accuracy.
Performance and Results
Our analysis across seven major polarized events—including political debates and sports championships—revealed several key findings.
Methodological Superiority
The proposed Unigram method significantly outperformed standard supervised learning models, such as Naive Bayes and Bagged Trees, particularly when applied to "emerging events" where prior labeled data is scarce.
In Figure 4, we observe that the Unigram method maintains a much higher correlation with human partiality ratings across different score bins compared to baseline models.
The Virality of Partiality
The most striking result is the "virality gap." Partial messages are twice as likely to be retweeted as impartial ones. This holds true even after controlling for the author's follower count and the specific topic being discussed, suggesting that bias itself is a primary driver of engagement on social media.
Temporal Dynamics
Impartiality is not a static property of a discussion. During the 2013 government shutdown, discourse began relatively neutral but became sharply more partial during key legislative moments, such as the proposal of major amendments.
Conclusion and Future Work
This work provides a new lens for studying online polarization by focusing on the impartiality of individual messages rather than just author identity. While our current model is robust, it faces limitations with linguistic nuances like sarcasm and irony, which can confuse unigram-based detection. Future research will explore more complex linguistic features, such as discourse markers and hedges, to better capture the subtleties of impartial speech in multi-factional real-world conflicts.
Summary updated