"We won't even challenge their lefty academic definition of racist: " Understanding the Use of e-Prints on Reddit and 4chan

Satrio Baskoro Yudhoatmojo, Emiliano De Cristofaro, Jeremy Blackburn

This paper explores how non-peer-reviewed scientific "e-prints" (preprints) are shared and discussed within two distinct online ecosystems: the mainstream, multi-faceted Reddit and the fringe, often extremist 4chan’s Politically Incorrect board (/pol/). By analyzing over a decade of data, the researchers uncover how these early-stage scientific findings are often misinterpreted, generalized, or weaponized to support specific community narratives, particularly during the COVID-19 pandemic.

TL;DR

The study analyzes 14 years of Reddit and 4 years of 4chan data to track the dissemination of non-peer-reviewed scientific e-prints. It finds that while these platforms facilitate open science, they also enable the frequent misinterpretation of research by laypeople, who often use papers as "props" for arguments rather than engaging with the full text.

What Problem Does This Paper Address?

The traditional scientific publishing model acts as a gatekeeper, ensuring peer review but often at the cost of speed and public access. e-Print servers like arXiv and bioRxiv have solved the access problem, allowing researchers to share findings instantly. However, this "raw" science lacks the quality control of peer review.

The authors address the concern that the general public—lacking specialized training—may struggle to interpret these preliminary findings. This is especially critical in "fringe" communities where scientific authority might be viewed with skepticism or used selectively to bolster ideological positions. The paper seeks to quantify the presence of e-prints in these communities and understand the nature of the surrounding discourse.

Methods

The researchers employed a multi-stage, data-driven approach:

  • Link Extraction: They filtered massive datasets from Reddit and 4chan for links to eight major e-print servers (e.g., arXiv, bioRxiv, medRxiv, viXra).
  • Topic Modeling: They used Top2Vec, an algorithm that leverages Doc2Vec and Word2Vec to create joint word and document embeddings, to automatically identify the research themes of the shared e-prints.
  • Similarity Analysis: To measure how much the online discussion actually reflected the paper's content, they used Doc2Vec to generate embeddings for both the e-prints (abstracts and full text) and the social media threads. They then calculated the cosine similarity between these embeddings.
  • Qualitative Case Studies: The authors manually reviewed high-engagement threads to identify specific patterns of use, such as debunking, speculation, or misinterpretation.

Datasets

  • Reddit: 14 years of submission data (June 2005 – March 2021) and comment data (Dec 2005 – March 2021) sourced from Pushshift.
  • 4chan /pol/: Over 4 years of thread data (June 2016 – March 2021).
  • e-Prints: A curated dataset of 54,684 unique e-prints linked across the two platforms.

Key Insights and Ideas

  • The "Abstract-Only" Engagement: Similarity scores were consistently higher between discussion threads and paper abstracts than with the full text. This suggests that users rarely read beyond the summary before posting.
  • COVID-19 as a Catalyst: The pandemic caused a massive surge in links to medical and biological e-prints (medRxiv and bioRxiv), especially on 4chan's /pol/. In these early months, preliminary findings were frequently used to speculate on infection rates and mortality before they could be properly vetted.
  • Scientific "Props": The overall low similarity between threads and e-prints (< 0.32 on average) suggests that papers are often used as rhetorical "props." Instead of discussing the nuances of the research, users cite the paper to lend a veneer of scientific authority to their existing arguments.
  • Community Self-Policing: On Reddit, science-enthusiast communities (like r/science) were observed actively debunking "pseudoscience" e-prints, particularly those from viXra, which has lower moderation standards than arXiv.
Temporal trends show a massive spike in e-print links during the onset of the COVID-19 pandemic.

Key Results

  • Volume: arXiv is the dominant server on Reddit (50,904 e-prints). On /pol/, medRxiv and bioRxiv are disproportionately popular, driven by COVID-19 discussions.
  • Concentration: About half of the subreddits in the dataset contained only a single e-print link, indicating that while e-prints are widespread, deep engagement is concentrated in specific communities.
  • Quantitative Similarity: As shown in Figure 3, the similarity between threads and the e-prints they link to is generally low, reinforcing the idea that the discussions often drift far from the actual scientific content.
The CDF of similarity scores shows that discussions rarely align closely with the full text of the research.

Limitations and Future Work

  • Server Scope: The study focused on eight specific e-print servers; it did not include open-access but peer-reviewed repositories like PLOS or MDPI.
  • Peer-Review Tracking: The authors did not track which e-prints eventually passed peer review, which could provide insight into whether "bad" science is more likely to be shared in fringe communities.
  • Platform Breadth: The study was limited to Reddit and 4chan. Future work could expand to Twitter or Facebook to see if these patterns hold in more mainstream social networks.
  • Call to Action: The authors suggest that scientists should include "simple English" summaries with their e-prints to help mitigate misinterpretation by the general public.

Summary updated

Abstract

The dissemination and reach of scientific knowledge have increased at a blistering pace. In this context, e-Print servers have played a central role by providing scientists with a rapid and open mechanism for disseminating research without waiting for the (lengthy) peer review process. While helping the scientific community in several ways, e-Print servers also provide scientific communicators and the general public with access to a wealth of knowledge without paying hefty subscription fees. This motivates us to study how e-Prints are positioned within Web community discussions. In this paper, we analyze data from two Web communities: 14 years of Reddit data and over 4 from 4chan's Politically Incorrect board. Our findings highlight the presence of e-Prints in both science-enthusiast and general-audience communities. Real-world events and distinct factors influence the e-Prints people's discussions; e.g., there was a surge of COVID-19-related research publications during the early months of the outbreak and increased references to e-Prints in online discussions. Text in e-Prints and in online discussions referencing them has a low similarity, suggesting that the latter are not exclusively talking about the findings in the former. Further, our analysis of a sample of threads highlights: 1) misinterpretation and generalization of research findings, 2) early research findings being amplified as a source for future predictions, and 3) questioning findings from a pseudoscientific e-Print. Overall, our work emphasizes the need to quickly and effectively validate non-peer-reviewed e-Prints that get substantial press/social media coverage to help mitigate wrongful interpretations of scientific outputs.
BibTeX
@inproceedings{yudhoatmojo2023we,
  author = {Satrio Baskoro Yudhoatmojo and Emiliano De Cristofaro and Jeremy Blackburn},
  title = {{"We won't even challenge their lefty academic definition of racist: " Understanding the Use of e-Prints on Reddit and 4chan}},
  booktitle = {WebSci},
  year = {2023},
  url = {https://openreview.net/forum?id=KpTiQJt40V},
}
Figure