Crowd View: Converting Investors' Opinions into Indicators

Chung-Chi Chen, Hen-Hsen Huang, Hsin-Hsi ChenIJCAI 2019 · DOI

TL;DR

This paper introduces "Crowd View," a system that extracts fine-grained financial indicators (beyond simple sentiment) from real-time Twitter data. These indicators, like support/resistance prices and buy/sell costs, offer valuable insights for traders, complementing traditional market analysis.

What Problem Does This Paper Address?

The paper addresses the diminishing effectiveness of traditional sentiment analysis for predicting stock market movements. While early research showed a correlation between social media sentiment and stock prices, recent studies suggest this signal is weakening. The authors argue that simply knowing if people are "bullish" or "bearish" isn't enough; traders need more nuanced information. They propose that extracting specific financial opinions from crowd investors, such as price targets, support/resistance levels, and cost bases, can provide a richer, more predictive signal for trading decisions.

Methods

The "Crowd View" system operates in several stages. First, it monitors Twitter using the tweepy library to collect real-time tweets containing cashtags for Dow Jones Industrial Average (DJI) component stocks. Next, it employs a novel "OI extractor" to identify and extract numerical data related to five specific "opinion indicators" (OIs): support price level, resistance price level, price target, buy-side cost, and sell-side cost. The authors explored various CNN-based and RNN-based architectures for this extraction, finding that a word-based CNN architecture with pre-trained embeddings performed best. Finally, the extracted, structured data is visualized alongside real-time stock prices, with a 5-day moving average applied to smooth the OIs for easier analysis.

Datasets

The primary data source for this research is real-time textual information collected from Twitter. The paper also mentions using stock price data from Yahoo Finance. No new benchmark datasets were introduced; instead, the system leverages existing public data streams.

Key Insights and Ideas

  • Beyond Sentiment: The core idea is that "fine-grained opinions" from the crowd are more valuable than broad sentiment. This moves beyond a binary "good/bad" signal to a more quantitative understanding of investor expectations.
  • Twitter as a Financial Oracle: The paper demonstrates that informal, unstructured text on Twitter can be mined to derive structured financial indicators that are predictive.
  • Leading Indicators: The extracted OIs, particularly support/resistance levels and buy/sell costs, are shown to act as leading indicators for stock price movements, offering insights before the market fully reacts.
  • Complementary Information: The OIs provide information that is both comparable and complementary to traditional investor analysis, offering a unique perspective.

Key Results

The empirical studies indicate that the proposed OIs provide incremental information for trading. Specifically:

  • Support and resistance price levels offer valuable information for short-term (3- and 5-day) trading.
  • Buy-side and sell-side cost indicators provide additional long-term (10-day) trading information. The paper presents tables (Table 1 and Table 2) showing Kolmogorov-Smirnov (KS-test) results, which demonstrate statistically significant differences in conditional returns when the OI conditions are met, suggesting the OIs are informative.

Limitations and Future Work

The authors acknowledge that the effectiveness of the OIs depends on the conditions being met, and when distributions of conditional and unconditional returns are similar, the OIs may not add significant information. Future work suggested includes exploring the impact of these OIs on market volatility and financial derivatives like futures and options.

Summary updated

BibTeX
@inproceedings{chen2019crowd,
  author = {Chung-Chi Chen and Hen-Hsen Huang and Hsin-Hsi Chen},
  title = {{Crowd View: Converting Investors' Opinions into Indicators}},
  booktitle = {International Joint Conference on Artificial Intelligence},
  year = {2019},
  doi = {10.24963/ijcai.2019/936},
}
Figure