A Model of Online Misinformation

Daron Acemoğlu, Asuman Ozdaglar, James Siderius2021 · DOI

Abstract

We present a model of online content sharing where agents sequentially observe an article and must decide whether to share it with others. This content may or may not contain misinformation. Agents gain utility from positive social media interactions but do not want to be called out for propagating misinformation. We characterize the (Bayesian-Nash) equilibria of this social media game and show sharing exhibits strategic complementarity. Our first main result establishes that the impact of homophily on content virality is non-monotone: homophily reduces the broader circulation of an article, but it creates echo chambers that impose less discipline on the sharing of low-reliability content. This insight underpins our second main result, which demonstrates that social media platforms interested in maximizing engagement tend to design their algorithms to create more homophilic communication patterns ("filter bubbles"). We show that platform incentives to amplify misinformation are particularly pronounced for low-reliability content likely to contain misinformation and when there is greater polarization and more divisive content. Finally, we discuss various regulatory solutions to such platform-manufactured misinformation.

Introduction

Despite the global concern over "fake news," formal frameworks to understand the interplay between individual sharing behavior, network structure, and platform algorithms have been scarce. This paper provides a game-theoretic framework to explain why misinformation thrives on social media, even among rational actors. It reveals a "non-monotone" effect of homophily: while connecting like-minded people limits the broad reach of high-quality news, it creates "echo chambers" that shield users from reputational costs, actively encouraging the viral spread of low-reliability content.

Theoretical Framework: The Social Media Game

The authors develop a mathematical model inhabited by $N$ fully Bayesian agents. In this model, agents sequentially observe an article and must choose one of three actions: Share, Ignore, or Dislike (calling out the content).

Figure 1: An illustration of the social media game where agents choose to Share, Ignore, or Dislike content.

The utility function for these agents is driven by two factors:

  1. Positive Engagement: Agents gain utility when their shares are re-shared by others.
  2. Reputational Cost: Agents incur a cost if they are "disliked" or called out by others for sharing misinformation.

Articles are defined by their reliability (the probability of being true), their message (e.g., ideological bias), and their actual veracity. To represent communities, the authors use "island networks," allowing them to adjust the level of homophily—the likelihood of being connected to someone with the same ideological bias. As a theoretical paper, it relies on mathematical proofs rather than empirical datasets, though it is grounded in observed social media behaviors.

Key Insights: The Homophily Paradox

The model reveals several critical dynamics of online sharing:

  • Strategic Complementarity: Sharing is contagious. If an agent believes others are likely to share a post, the potential for positive engagement increases while the relative risk of being the sole person "called out" decreases, incentivizing further sharing.
  • The Homophily Paradox: Homophily has a split effect. For high-reliability news, homophily reduces virality because the news remains trapped within a single "island." However, for low-reliability news, homophily increases virality by creating a "safe space" where users do not fear being disciplined by those with opposing views.
  • Platform-Manufactured Misinformation: A platform seeking to maximize engagement will actively design filter bubbles for low-reliability content. By keeping questionable news within ideologically aligned groups, the platform ensures it spreads virally rather than being suppressed by cross-cutting "dislikes."

Main Results

The theoretical findings are formalized in several key theorems:

  • Theorem 2 (Non-monotone Homophily): Establishes a reliability threshold. Below this threshold (unreliable content), increasing homophily increases virality. Above it (reliable content), increasing homophily decreases virality.
  • Theorem 3 (Optimal Algorithms): To maximize engagement, a platform will choose maximal connectivity (low homophily) for reliable news but maximal homophily (filter bubbles) for unreliable news.
  • Proposition 4: Increased political polarization strengthens the platform's incentive to create filter bubbles for low-reliability content.

Discussion and Limitations

The model also highlights the "Implied Truth" effect: regulatory censorship can backfire. If a platform censors some misinformation, Bayesian users may assume anything not censored is true, accelerating the spread of misinformation that evades detection.

While the model assumes perfectly rational Bayesian agents, real-world cognitive limitations like confirmation bias likely exacerbate these effects. Future work could explore bounded rationality or dynamic models where user reputations and political views evolve over time based on their interaction history.

Summary updated

Abstract

We present a model of online content sharing where agents sequentially observe an article and must decide whether to share it with others.This content may or may not contain misinformation.Agents gain utility from positive social media interactions but do not want to be called out for propagating misinformation.We characterize the (Bayesian-Nash) equilibria of this social media game and show sharing exhibits strategic complementarity.Our first main result establishes that the impact of homophily on content virality is non-monotone: homophily reduces the broader circulation of an article, but it creates echo chambers that impose less discipline on the sharing of low-reliability content.This insight underpins our second main result, which demonstrates that social media platforms interested in maximizing engagement tend to design their algorithms to create more homophilic communication patterns ("filter bubbles").We show that platform incentives to amplify misinformation are particularly pronounced for low-reliability content likely to contain misinformation and when there is greater polarization and more divisive content.Finally, we discuss various regulatory solutions to such platform-manufactured misinformation.
BibTeX
@misc{acemoglu2021model,
  author = {Daron Acemoğlu and Asuman Ozdaglar and James Siderius},
  title = {{A Model of Online Misinformation}},
  year = {2021},
  doi = {10.3386/w28884},
}
Figure