Wisdom of the Crowds or Ignorance of the Masses? A data-driven guide to WSB

Valentina Semenova, Dragos Gorduza, William Wildi, Xiaowen Dong, Stefan ZohrenarXiv 2023 · DOI · arXiv

TL;DR

This paper provides a comprehensive, data-driven autopsy of the r/WallStreetBets (WSB) phenomenon, using large language models and network analysis to map the forum's evolution. The researchers find that while WSB is largely a "reactive" community of trend-followers—especially regarding "meme stocks"—small, niche sub-clusters within the forum may actually possess genuine market insights.

What Problem Does This Paper Address?

The 2021 GameStop short squeeze turned the financial world's eyes toward a corner of Reddit: r/WallStreetBets. Economists and investors were left asking: Is this the "wisdom of the crowds" or just the "ignorance of the masses"?

While previous studies looked at specific events like the GME squeeze, this paper seeks to characterize the forum's behavior over a decade (2012–2022). The authors tackle three main questions:

  1. What are the primary topics of interest, and how do they evolve?
  2. Can forum discussions actually predict market moves, or are they just noise?
  3. Are WSB users leading the market or simply chasing "hype" after the price has already moved?

Methods

The authors employed a multi-stage machine learning and econometric pipeline:

  • Topic Modeling: They used a BERT-based Large Language Model (LLM) to extract semantic topics, moving beyond simple keyword counting to understand the context of discussions.
  • Sentiment Analysis: They fine-tuned FinBERT (a pre-trained model for financial text) on a manually labeled dataset of 4,000 WSB posts. Crucially, they labeled for "future-facing" sentiment (bullish/bearish) rather than just general mood.
  • Network Analysis: They constructed two types of networks: a Topic Network (linking assets discussed within the same themes) and a Submission Network (linking assets based on shared user interest). They used the Leiden algorithm to identify communities of stocks.
  • Event Studies: They calculated Cumulative Abnormal Returns (CAR) using the Capital Asset Pricing Model (CAPM) to see how stock prices behaved in the 14 days surrounding a post.
  • Causality Testing: They applied Granger causality tests to determine if changes in forum sentiment statistically precede changes in asset returns.

Datasets

  • Reddit Data: 202,681 submissions and over 6.3 million comments from the Pushshift Reddit database (April 2012 – June 2022).
  • Market Data: Daily adjusted close prices from AlphaVantage for stocks mentioned on the forum.
  • New Benchmarks: The authors introduced a hand-annotated dataset of "Due Diligence" (DD) posts and a sentiment-labeled corpus specifically tailored to WSB's unique jargon.

Key Insights and Ideas

  • The "Hype" Reversal: For most popular stocks, WSB activity is reactive. The average stock price runs up before the post and crashes immediately after. This suggests users are trend-followers who often "buy the top."
  • Niche vs. Noise: The researchers found that "meme stock" clusters (GME, AMC) are the most reactive and least predictive. However, smaller, niche clusters (e.g., those focused on specific sectors like biotech or aerospace) showed statistically significant positive returns, suggesting "pockets" of well-informed investors.
  • The 2021 Contrarian Signal: Interestingly, in 2021, bullish "Due Diligence" posts were actually a negative predictor of returns. Following the advice of the most hyped posts during the peak of the craze was a losing strategy.
  • Persistent vs. Sporadic Interest: While topics like the S&P 500 are evergreen, others like "COVID" or "Oil" are transient. Tesla (TSLA) stands out as a unique "meme" topic that has maintained high interest for years.

Key Results

  • Sentiment Accuracy: The fine-tuned FinBERT achieved 69.2% accuracy in predicting future-facing sentiment—a high mark given the forum's heavy use of sarcasm and slang.
  • Granger Causality: Sentiment was found to Granger-cause returns for "meme" stocks like GME, AMC, NOK, and BB, as well as SNAP. For most "blue-chip" stocks (AAPL, MSFT), no such causal link exists.
  • Portfolio Performance: A portfolio following all WSB mentions yielded an average next-day log-return of -0.0177 (a losing strategy). However, "Due Diligence" posts flaired by moderators showed positive, statistically significant returns (0.0074).
  • Sentiment Skew: The forum is overwhelmingly optimistic; 77% of DD posts are bullish.

Limitations and Future Work

The authors acknowledge several hurdles:

  • Data Imbalance: Over 60% of the total post volume occurred in 2021 alone, which may skew long-term historical analysis.
  • Sarcasm and Slang: Despite using FinBERT, the "ironic" nature of WSB (e.g., users calling themselves "apes" or "regards") remains a challenge for NLP.
  • Moderation Bias: The study relies on posts that survived moderation; deleted or removed posts (which might contain different signals) were not fully captured.
  • Future Directions: The authors suggest further research into "non-stationary" datasets—how the predictive power of a community shifts as it grows from a niche group to a mainstream phenomenon.

Summary updated

Abstract

A trite yet fundamental question in economics is: What causes large asset price fluctuations? A tenfold rise in the price of GameStop equity, between the 22nd and 28th of January 2021, demonstrated that herding behaviour among retail investors is an important contributing factor. This paper presents a data-driven guide to the forum that started the hype -- WallStreetBets (WSB). Our initial experiments decompose the forum using a large language topic model and network tools. The topic model describes the evolution of the forum over time and shows the persistence of certain topics (such as the market / S\&P500 discussion), and the sporadic interest in others, such as COVID or crude oil. Network analysis allows us to decompose the landscape of retail investors into clusters based on their posting and discussion habits; several large, correlated asset discussion clusters emerge, surrounded by smaller, niche ones. A second set of experiments assesses the impact that WSB discussions have had on the market. We show that forum activity has a Granger-causal relationship with the returns of several assets, some of which are now commonly classified as `meme stocks', while others have gone under the radar. The paper extracts a set of short-term trade signals from posts and long-term (monthly and weekly) trade signals from forum dynamics, and considers their predictive power at different time horizons. In addition to the analysis, the paper presents the dataset, as well as an interactive dashboard, in order to promote further research.
BibTeX
@misc{semenova2023wisdom,
  author = {Valentina Semenova and Dragos Gorduza and William Wildi and Xiaowen Dong and Stefan Zohren},
  title = {{Wisdom of the Crowds or Ignorance of the Masses? A data-driven guide to WSB}},
  year = {2023},
  eprint = {2308.09485},
  archivePrefix = {arXiv},
  url = {https://arxiv.org/abs/2308.09485},
}
Figure