This paper explores how non-peer-reviewed scientific "e-prints" (preprints) are shared and discussed within two distinct online ecosystems: the mainstream, multi-faceted Reddit and the fringe, often extremist 4chan’s Politically Incorrect board (/pol/). By analyzing over a decade of data, the researchers uncover how these early-stage scientific findings are often misinterpreted, generalized, or weaponized to support specific community narratives, particularly during the COVID-19 pandemic.
TL;DR
The study analyzes 14 years of Reddit and 4 years of 4chan data to track the dissemination of non-peer-reviewed scientific e-prints. It finds that while these platforms facilitate open science, they also enable the frequent misinterpretation of research by laypeople, who often use papers as "props" for arguments rather than engaging with the full text.
What Problem Does This Paper Address?
The traditional scientific publishing model acts as a gatekeeper, ensuring peer review but often at the cost of speed and public access. e-Print servers like arXiv and bioRxiv have solved the access problem, allowing researchers to share findings instantly. However, this "raw" science lacks the quality control of peer review.
The authors address the concern that the general public—lacking specialized training—may struggle to interpret these preliminary findings. This is especially critical in "fringe" communities where scientific authority might be viewed with skepticism or used selectively to bolster ideological positions. The paper seeks to quantify the presence of e-prints in these communities and understand the nature of the surrounding discourse.
Methods
The researchers employed a multi-stage, data-driven approach:
- Link Extraction: They filtered massive datasets from Reddit and 4chan for links to eight major e-print servers (e.g., arXiv, bioRxiv, medRxiv, viXra).
- Topic Modeling: They used Top2Vec, an algorithm that leverages Doc2Vec and Word2Vec to create joint word and document embeddings, to automatically identify the research themes of the shared e-prints.
- Similarity Analysis: To measure how much the online discussion actually reflected the paper's content, they used Doc2Vec to generate embeddings for both the e-prints (abstracts and full text) and the social media threads. They then calculated the cosine similarity between these embeddings.
- Qualitative Case Studies: The authors manually reviewed high-engagement threads to identify specific patterns of use, such as debunking, speculation, or misinterpretation.
Datasets
- Reddit: 14 years of submission data (June 2005 – March 2021) and comment data (Dec 2005 – March 2021) sourced from Pushshift.
- 4chan /pol/: Over 4 years of thread data (June 2016 – March 2021).
- e-Prints: A curated dataset of 54,684 unique e-prints linked across the two platforms.
Key Insights and Ideas
- The "Abstract-Only" Engagement: Similarity scores were consistently higher between discussion threads and paper abstracts than with the full text. This suggests that users rarely read beyond the summary before posting.
- COVID-19 as a Catalyst: The pandemic caused a massive surge in links to medical and biological e-prints (medRxiv and bioRxiv), especially on 4chan's /pol/. In these early months, preliminary findings were frequently used to speculate on infection rates and mortality before they could be properly vetted.
- Scientific "Props": The overall low similarity between threads and e-prints (< 0.32 on average) suggests that papers are often used as rhetorical "props." Instead of discussing the nuances of the research, users cite the paper to lend a veneer of scientific authority to their existing arguments.
- Community Self-Policing: On Reddit, science-enthusiast communities (like r/science) were observed actively debunking "pseudoscience" e-prints, particularly those from viXra, which has lower moderation standards than arXiv.
Key Results
- Volume: arXiv is the dominant server on Reddit (50,904 e-prints). On /pol/, medRxiv and bioRxiv are disproportionately popular, driven by COVID-19 discussions.
- Concentration: About half of the subreddits in the dataset contained only a single e-print link, indicating that while e-prints are widespread, deep engagement is concentrated in specific communities.
- Quantitative Similarity: As shown in Figure 3, the similarity between threads and the e-prints they link to is generally low, reinforcing the idea that the discussions often drift far from the actual scientific content.
Limitations and Future Work
- Server Scope: The study focused on eight specific e-print servers; it did not include open-access but peer-reviewed repositories like PLOS or MDPI.
- Peer-Review Tracking: The authors did not track which e-prints eventually passed peer review, which could provide insight into whether "bad" science is more likely to be shared in fringe communities.
- Platform Breadth: The study was limited to Reddit and 4chan. Future work could expand to Twitter or Facebook to see if these patterns hold in more mainstream social networks.
- Call to Action: The authors suggest that scientists should include "simple English" summaries with their e-prints to help mitigate misinterpretation by the general public.
Summary updated