Exploring agent interaction patterns in the comment sections of fake and real news

Kailun Zhu, Songtao Peng, Jiaqi Nie, Zhongyuan Ruan, Shanqing Yu, Qi Xuan

Abstract

User comments on social media have been recognized as a crucial factor in distinguishing between fake and real news, with many studies focusing on the textual content of user reactions. However, the interactions among agents in the comment sections for fake and real news have not been fully explored. In this study, we analyze a dataset comprising both fake and real news from Reddit to investigate agent interaction patterns, considering both the network structure and the sentiment of the nodes. Our findings reveal that (i) comments on fake news are more likely to form groups, (ii) compared to fake news, where users generate more negative sentiment, real news tend to elicit more neutral and positive sentiments. Additionally, nodes with similar sentiments cluster together more tightly than anticipated. From a dynamic perspective, we found that the sentiment distribution among nodes stabilizes early and remains stable over time. These findings have both theoretical and practical implications, particularly for the early detection of real and fake news within social networks.

Introduction

While traditional fake news detection focuses on linguistic features or user metadata, the social dynamics within comment sections remain under-explored. This research investigates how interaction patterns—the "shape" and "vibe" of conversations—differ when users engage with verified versus fabricated information on Reddit. By modeling these discussions as complex networks, we can identify structural and emotional signals that distinguish fake news from real news.

Methodology

The study analyzed 659 real news posts and 235 fake news posts from Reddit, totaling over 136,000 comments. Discussions were modeled as Comment Trees, where replies form branches from the original post. Using the pysentimiento BERT-based model, each comment was classified by sentiment (positive, neutral, negative) and specific emotions (joy, sadness, anger, disgust, fear).

To validate findings, the researchers used Null Models, comparing empirical data against randomized versions where sentiment labels were shuffled. Finally, a Dynamic Analysis tracked the evolution of these threads across five stages of growth (from 20% to 100% of the final comment count).

Key Findings

1. The Negativity Magnet

A primary distinction between the two news types is the emotional response they provoke. Fake news acts as a significant magnet for negative sentiment. Specifically, fake news threads exhibit higher proportions of disgust and anger. In contrast, real news discussions are characterized by higher levels of neutral and positive sentiments, including joy.

Figure 2: Comparison of sentiment polarities (a) and specific emotions (b) between fake and real news. Fake news shows significantly higher negative sentiment and emotions like disgust and anger.

2. Early Emotional Imprinting and Stability

One of the study's most significant findings is the temporal stability of a thread's emotional tone. The "vibe" of a discussion is established almost immediately. The sentiment distribution observed in the first 20% of comments (Stage 1) remains remarkably consistent throughout the entire lifespan of the thread. This suggests that the initial reaction to a post is a powerful predictor of its long-term social dynamic.

Figure 3: Sentiment (a, b) and emotion (c, d) proportions across five stages of thread growth. The distributions remain stable from the earliest stage (S1) to the final stage (S5).

3. Emotional Homophily

The research also identified a "birds of a feather" effect in user interactions. Users do not reply at random; they tend to cluster with others expressing similar sentiments. This emotional homophily was particularly strong for negative comments, where clustering was 27% higher than expected by chance.

Conclusion

These interaction patterns provide valuable signals for early fake news detection. The fact that sentiment stabilizes early suggests that automated systems could potentially flag suspicious content shortly after it begins to go viral, based on the community's emotional reaction rather than just the content itself. Future work will focus on integrating these network-sentiment features into machine learning classifiers to improve detection speed and accuracy.

Summary updated

Abstract

User comments on social media have been recognized as a crucial factor in distinguishing between fake and real news, with many studies focusing on the textual content of user reactions. However, the interactions among agents in the comment sections for fake and real news have not been fully explored. In this study, we analyze a dataset comprising both fake and real news from Reddit to investigate agent interaction patterns, considering both the network structure and the sentiment of the nodes. Our findings reveal that (i) comments on fake news are more likely to form groups, (ii) compared to fake news, where users generate more negative sentiment, real news tend to elicit more neutral and positive sentiments. Additionally, nodes with similar sentiments cluster together more tightly than anticipated. From a dynamic perspective, we found that the sentiment distribution among nodes stabilizes early and remains stable over time. These findings have both theoretical and practical implications, particularly for the early detection of real and fake news within social networks.
BibTeX
@article{zhu2024exploring,
  author = {Kailun Zhu and Songtao Peng and Jiaqi Nie and Zhongyuan Ruan and Shanqing Yu and Qi Xuan},
  title = {{Exploring agent interaction patterns in the comment sections of fake and real news}},
  journal = {Journal of The Royal Society Interface},
  year = {2024},
  doi = {10.1098/rsif.2024.0483},
}
Figure