Dynamic Simulation Framework for Disinformation Dissemination and Correction With Social Bots

Boyu Qiao, Kun Li, Wei Zhou, Songlin Hu

TL;DR

This paper introduces MADD, a novel multi-agent simulation framework that models the complex interplay of social bots spreading and correcting disinformation in online ecosystems. By integrating realistic network structures and user attributes, MADD offers a more nuanced understanding of disinformation dynamics and provides a platform for evaluating correction strategies.

What Problem Does This Paper Address?

The proliferation of disinformation on social media, amplified by sophisticated social bots, poses a significant challenge to information integrity and societal discourse. While existing research has explored disinformation dissemination, many models oversimplify user behavior, network structures, and the dynamic role of both malicious and legitimate bots. Specifically, there's a need for a framework that can:

  • Model realistic social network structures: Beyond simple network topologies, real-world networks exhibit both scale-free properties (influential hubs) and community structures.
  • Incorporate nuanced user attributes: User behavior is influenced by factors like interest communities, trust thresholds, and social influence, which are often overlooked.
  • Simulate the dynamic interplay of malicious and legitimate bots: Understanding how bots actively spread disinformation and how other bots (or human users) attempt to correct it is crucial.
  • Quantitatively evaluate correction strategies: The effectiveness of different approaches to combating disinformation needs rigorous assessment.

Methods

The authors propose MADD (Multi-Agent-based framework for Disinformation Dissemination), a comprehensive simulation framework built upon several key components:

  • Realistic Network Construction: MADD combines the Barabási-Albert Model (BAM) for scale-free topology with the Stochastic Block Model (SBM) to capture community structures, creating a more realistic representation of social networks.
  • Attribute-Driven Agents: Agents (users, malicious bots, legitimate bots) are endowed with five core attributes:
  • Interest Community (IC): Reflects a user's engagement with specific topics.
  • Trust Threshold (TT): Measures a user's susceptibility to disinformation.
  • Dissemination Tendency (DT): Quantifies a user's likelihood to share information, influenced by IC and a power-law distribution.
  • Social Influence (SI): Represents a user's centrality in the network.
  • Activation Time (AT): Models the temporal dynamics of user participation.
  • Disinformation and Correction Rules: Disinformation is characterized by topic and plausibility. Correction strategies are categorized into fact-based (citing evidence) and narrative-based (using stories/metaphors). Intervention timing (early, mid, late) is also considered.
  • Dynamic Trust Threshold Model: A novel approach models the evolution of user trust thresholds based on exposure to both disinformation and corrective content, incorporating enhancement and decay mechanisms.
  • LLM Integration: Large Language Models (LLMs) are used for evaluating user attributes (e.g., interest communities, trust thresholds, disinformation plausibility) and for simulating user decision-making during information sharing.

Datasets

  • Real-world User Data: Approximately 100 regular users from six distinct communities (Entertainment, Technology, Sports, Business, Politics, Education) on the X/Twitter platform were sampled. Data included basic profile information (username, self-introduction, post count, follower count, etc.) and a sample of 200 content posts (original posts, retweets, quotes) collected over a three-day period.
  • Disinformation Topics: Six disinformation topics were used, sourced from verified cases by Snopes, PolitiFact, and FactCheck.org.

Key Insights and Ideas

  • The importance of network structure: MADD's hybrid network model (BAM + SBM) is crucial for capturing the complex dynamics of information spread, which differs significantly from simpler network models.
  • Dynamic trust is key: User trust thresholds are not static; they evolve based on exposure to information, highlighting the need for dynamic modeling rather than fixed susceptibility.
  • Early intervention is critical: The effectiveness of correction strategies diminishes significantly when applied in mid or late stages of disinformation spread, suggesting a narrow window for impactful intervention.
  • Fact-based correction is generally more effective: While narrative-based correction has its place, fact-based approaches tend to yield larger increases in user trust thresholds across most communities.
  • "Politics" is a challenging domain: This community exhibits unique patterns of disinformation spread and resistance to correction, likely due to its inherent subjectivity and the prevalence of echo chambers.
  • LLMs can enhance realism but introduce biases: While LLMs are powerful for simulating nuanced user behavior, their potential topic-specific biases need careful consideration in future work.

Key Results

  • Real-world consistency: MADD's simulated user attributes and network structures statistically align with real-world social network characteristics (e.g., heavy-tailed degree distributions, normal trust threshold distributions).
  • Disinformation spread patterns: Simulations show typical rapid initial increases in "infected" spreaders followed by a decline, mirroring epidemic models and empirical studies.
  • Correction strategy effectiveness:
    • Early intervention with fact-based correction significantly reduced disinformation spread in "Business," "Politics," and "Technology."
    • Mid and late-stage interventions showed limited effectiveness, and in some cases, negative impacts (increased spread) in communities like "Politics" and "Business."
    • Legitimate bot ratio: Increasing the proportion of legitimate bots (from 5% to 30%) led to faster and more significant declines in infection rates during the middle stages of intervention.
  • Trust threshold dynamics: Fact-based correction generally led to higher trust thresholds compared to the control group, especially during early intervention.
  • Computational efficiency: The proposed dynamic trust threshold calculation method achieved a significant reduction in token consumption compared to LLM-based evaluation while maintaining comparable accuracy.

Limitations and Future Work

  • Limited scope of simulation: Experiments were primarily conducted within single communities due to computational costs, rather than across the entire network. Future work should scale up to larger network models.
  • LLM biases: The potential for LLM topic-specific biases to influence simulation outcomes is acknowledged. Future research should prioritize more neutral topics or explore bias mitigation techniques.
  • Dataset size and duration: The dataset was relatively small (six communities, three days of activity), limiting the capture of long-term user preferences.
  • Future directions: The authors suggest extending the framework to larger-scale network models, exploring more diverse correction strategies, and investigating the impact of LLM biases more thoroughly.

Summary updated

Abstract

In the human-bot symbiotic information ecosystem, social bots play key roles in spreading and correcting disinformation. Understanding their influence is essential for risk control and better governance. However, current studies often rely on simplistic user and network modeling, overlook the dynamic behavior of bots, and lack quantitative evaluation of correction strategies. To fill these gaps, we propose MADD, a Multi Agent based framework for Disinformation Dissemination. MADD constructs a more realistic propagation network by integrating the Barabasi Albert Model for scale free topology and the Stochastic Block Model for community structures, while designing node attributes based on real world user data. Furthermore, MADD incorporates both malicious and legitimate bots, with their controlled dynamic participation allows for quantitative analysis of correction strategies. We evaluate MADD using individual and group level metrics. We experimentally verify the real world consistency of MADD user attributes and network structure, and we simulate the dissemination of six disinformation topics, demonstrating the differential effects of fact based and narrative based correction strategies.
BibTeX
@inproceedings{qiao2025dynamic,
  author = {Boyu Qiao and Kun Li and Wei Zhou and Songlin Hu},
  title = {{Dynamic Simulation Framework for Disinformation Dissemination and Correction With Social Bots}},
  booktitle = {Conference on Empirical Methods in Natural Language Processing},
  year = {2025},
}
Figure