Identifying and Characterizing Active Citizens who Refute Misinformation in Social Media

Yida Mu, Pu Niu, Nikolaos Aletras

Abstract

While the spread of misinformation on social media is a well-documented phenomenon, the individuals who actively work to combat it—referred to here as "active citizens"—remain understudied. This paper presents a comprehensive study aimed at automatically identifying and characterizing these users across different platforms (Twitter and Weibo) and languages (English and Chinese). We introduce a new large-scale Weibo dataset, evaluate a suite of hierarchical transformer-based models, and provide a detailed linguistic analysis of the differences between misinformation spreaders and those who refute them. Our results demonstrate that hierarchical models can identify active citizens with up to 85% accuracy, revealing that these users often employ more analytical and tentative language than their counterparts.

Introduction

Research into social media misinformation has traditionally focused on the mechanics of how false information spreads or on the automated detection of specific "fake news" posts. However, a critical component of the social media ecosystem is the human element: the users who act as a manual line of defense. These "active citizens" dedicate effort to reporting suspicious content or providing evidence-based refutations.

Understanding these users is vital for computational social science, as it allows platforms to identify helpful contributors early, assists in psychological profiling of digital debunkers, and improves the recommendation of fact-checking resources. This study addresses the challenge of identifying these users at scale, moving beyond small-scale, single-platform analyses to a cross-lingual, cross-platform framework.

Methodology: Hierarchical Transformers

A significant technical hurdle in user-level classification is the sheer volume of data. Standard Natural Language Processing (NLP) models, such as BERT, are typically limited to a 512-token input window. Because an active citizen's behavior is defined by their long-term history rather than a single post, a standard model would lose critical context.

To overcome this, we implemented a Hierarchical Transformer architecture:

  1. Chunking: A user’s entire post history is concatenated and divided into manageable chunks.
  2. Encoding: Each chunk is processed through a state-of-the-art transformer. We utilized RoBERTa for English (Twitter) and ERNIE for Chinese (Weibo) to capture deep semantic meanings.
  3. Fusion: To aggregate these individual chunk representations into a single "user embedding," we employed a Long Short-Term Memory (LSTM) network equipped with an attention mechanism. This allows the model to weigh certain periods of activity or specific types of posts more heavily than others.
  4. Classification: The final global embedding is passed through a classification layer to distinguish between misinformation posters and active citizens.

To ensure transparency, we applied InputXGrad, an explainability method that highlights which specific terms or linguistic markers most influenced the model’s classification decisions.

Data Collection and Categorization

The study utilizes two distinct datasets to test the cross-platform applicability of our models:

  • Weibo (Chinese): We developed a new dataset of 48,334 users. Active citizens were identified as users who reported posts to the Weibo Community Management Center that were subsequently verified and labeled as "false" by official moderators.
  • Twitter (English): We repurposed an existing dataset of 32,989 users. In this context, active citizens were defined as users who shared links to reputable fact-checking organizations (e.g., Snopes, PolitiFact) to refute circulating claims.

In both datasets, we focused on clear behavioral signals by filtering out "hybrid" users—those who both spread and refuted misinformation—who represented less than 10% of the total population.

Linguistic Insights and Characterization

Our analysis revealed several key differences in how active citizens communicate compared to misinformation spreaders:

  • Analytical and Tentative Tone: Across both platforms, active citizens tend to use more adverbs and words associated with cognitive processes and uncertainty (e.g., "perhaps," "confusing," "slightly"). This suggests a more cautious, analytical approach to information compared to the often certain and highly emotional tone of misinformation spreaders.
  • Topic Engagement: On Twitter, misinformation spreaders are frequently hyper-focused on specific global political events and figures. Active citizens, conversely, show engagement across a broader spectrum of social issues and daily discourse.
  • Platform-Specific Nuance: On Weibo, active citizens frequently use "net-speak" and informal language (e.g., "WTF," "LMAO," and various emojis). This indicates that on Weibo, the act of refuting misinformation is often performed by "average" users engaging in social interactions, rather than specialized or formal accounts.

Experimental Results

The hierarchical models significantly outperformed traditional baselines, such as Logistic Regression and standard BiLSTMs, proving that long-term linguistic history is essential for accurate identification.

  • Weibo Performance: The HierERNIE-LSTM model achieved a Macro F1 score of 85.1.
  • Twitter Performance: The HierLongformer-LSTM model achieved a Macro F1 score of 80.2.

The higher accuracy on Weibo may be attributed to the platform's centralized reporting mechanism, which provides a very clean signal for identifying active citizenship compared to the more decentralized nature of fact-check sharing on Twitter.

Conclusion and Future Work

This research provides a robust framework for identifying the "guardians" of digital discourse. By leveraging hierarchical transformers, we can capture the long-term behavioral patterns that define active citizenship in the face of misinformation. Future research will aim to move beyond binary classification to include "gray area" users, incorporate multi-modal data (such as images and videos), and explore cross-lingual transfer learning to identify active citizens in low-resource language environments without the need for massive new datasets.

Summary updated

Abstract

The phenomenon of misinformation spreading in social media has developed a new form of active citizens who focus on tackling the problem by refuting posts that might contain misinformation. Automatically identifying and characterizing the behavior of such active citizens in social media is an important task in computational social science for complementing studies in misinformation analysis. In this paper, we study this task across different social media platforms (i.e., Twitter and Weibo) and languages (i.e., English and Chinese) for the first time. To this end, (1) we develop and make publicly available a new dataset of Weibo users mapped into one of the two categories (i.e., misinformation posters or active citizens); (2) we evaluate a battery of supervised models on our new Weibo dataset and an existing Twitter dataset which we repurpose for the task; and (3) we present an extensive analysis of the differences in language use between the two user categories.
BibTeX
@inproceedings{mu2022identifying,
  author = {Yida Mu and Pu Niu and Nikolaos Aletras},
  title = {{Identifying and Characterizing Active Citizens who Refute Misinformation in Social Media}},
  booktitle = {WebSci},
  year = {2022},
  url = {https://openreview.net/forum?id=r69r4_EF6Z},
}
Figure