TL;DR
This paper presents a comprehensive traffic and network analysis of fake news websites, revealing their operational patterns, user engagement, and lifetime. The authors introduce a novel content-agnostic machine learning classifier for detecting fake news sites, achieving impressive accuracy.
What Problem Does This Paper Address?
The proliferation of fake news online poses a significant threat to society, making it difficult for individuals to discern truth from falsehood. While many studies focus on detecting fake news content or its spread on social media, there's a notable gap in understanding the underlying infrastructure and operational dynamics of fake news websites. This paper aims to fill this gap by investigating questions such as: How long do fake news websites typically stay online? Do they synchronize their activities? What kind of third-party services do they rely on? How do their traffic patterns compare to legitimate news sources? Understanding these aspects is crucial for developing more effective detection and mitigation strategies.
Methods
The researchers employed a multi-faceted approach:
- Data Collection: They compiled a dataset of 283 known fake news websites and a comparable set of real news websites. This was done by leveraging existing lists of fake news sites and top Alexa news sites, ensuring no overlap.
- Traffic and Audience Analysis: Using data from SimilarWeb and CheckPageRank, they analyzed user engagement metrics like visit duration, pages per visit, bounce rate, traffic sources, and website connectivity.
- Historical State Analysis: The Wayback Machine was used to collect historical snapshots of fake news websites over 23 years. A manual annotation process, aided by the Puppeteer framework, categorized website states as "alive" (offering news), "zombie" (offering non-news content), or "dead."
- Uptime and Content Synchronization: They analyzed time-series data of website states to identify patterns of synchronized uptime and content delivery among fake news sites, particularly around election periods.
- Third-Party Tracker Analysis: The study examined embedded third-party domains (like analytics and ad trackers) in fake news websites, comparing their prevalence to real news sites.
- Machine Learning Classification: Based on the collected network traffic features, they developed and evaluated various ML classifiers for automatic fake news website detection.
Datasets
- Fake News Websites: A dataset of 283 fake news domains, manually checked and flagged from the opensources.co repository.
- Real News Websites: A dataset of real news domains, selected from top Alexa news sites, ensuring no overlap with the fake news list.
- Historical Website Data: Archived website content from the Wayback Machine spanning over 20 years.
- Traffic Data: Web traffic and audience behavioral data from SimilarWeb and CheckPageRank.
- Third-Party Tracker Data: Information on embedded third-party trackers from Whotracks.me and AdblockPlus blacklists.
Key Insights and Ideas
- Synchronized Operations: Fake news sites exhibit a tendency to synchronize their uptime and content, especially during critical periods like elections. This suggests a coordinated effort rather than isolated operations.
- Purpose Beyond Monetization: During peak election periods, an increase in analytics usage was observed in fake news sites, but not a proportional increase in ad-related third-parties. This implies that during these times, the primary goal might have been polarization or misinformation dissemination rather than direct ad revenue.
- Distinct Third-Party Ecosystem: Fake news sites show a higher prevalence of certain third-party trackers (like Facebook and Quantserve) compared to real news sites, while real news sites rely more on others (like Google's DoubleClick and Scorecardresearch).
- Short Lifespans, but Peaks: While the median lifespan of fake news sites is relatively short (around 4 years), there are distinct peaks in their activity, notably around the 2016-2017 US presidential election, followed by a rapid decline.
- Content-Agnostic Detection: The research demonstrates that network traffic features alone can be highly effective for detecting fake news websites, even without analyzing the actual content.
Key Results
- User Engagement: Median real news sites receive more pages per visit (2.18) than fake news sites (1.72), and users spend longer on real news sites (198.8 seconds vs. 163.4 seconds).
- Bounce Rate: Real news sites have a significantly lower bounce rate (66.55%) compared to fake news sites (72.49%), indicating better visitor retention.
- Backlinks and Referring Domains: Fake news sites have drastically fewer backlinks (median 4.7K vs. 23.2M for real news) and referring domains (median 307 vs. 41.3K for real news), suggesting a lack of trust and authority.
- Lifespan and Activity: The median "alive time" for fake news sites is only 2 years, with a significant rise in activity observed during the 2016-2017 US presidential election.
- ML Classifier Performance: The developed content-agnostic ML classifier achieved a high F1 score of up to 0.942 and an AUC of ROC up to 0.976, demonstrating its effectiveness in distinguishing fake from real news websites.
Limitations and Future Work
- Limitations: The study acknowledges that the Wayback Machine does not archive all websites at all times, leading to "missing" timestamps. They used interpolation to address this, but it's an approximation. The manual annotation of website states is also a labor-intensive process.
- Future Work: The authors plan to investigate real-time updates for their ML model by collecting data at regular intervals or as new websites are discovered. They envision a crowd-sourced approach using browser plugins and explore privacy-preserving techniques like Federated Learning for model training.
Summary updated