TL;DR
This paper explores fine-tuning Large Language Models (LLMs) to predict stock returns directly from financial news. By comparing different LLM architectures and representation methods, they demonstrate that LLM-based predictions can significantly enhance portfolio construction, outperforming traditional sentiment analysis.
What Problem Does This Paper Address?
Quantitative investing relies on extracting predictive signals from various data sources to build optimal portfolios. While financial news has long been recognized as a valuable source of information for predicting stock performance, traditional methods involve a multi-step process of feature extraction (like sentiment analysis) and validation. This process can be time-consuming and require extensive manual effort. The paper addresses the challenge of directly leveraging the rich semantic understanding of LLMs for stock return prediction, aiming to streamline this process and potentially uncover more nuanced predictive signals than conventional methods.
Methods
The authors propose a novel LLM-based framework for direct news-to-return forecasting. This framework consists of two main modules: a text representation module and a forecasting module.
- Text Representation Module: They compare two types of LLMs for generating text representations:
- Encoder-only LLMs (e.g., DeBERTa): These models are pre-trained using masked language modeling, encouraging representations that capture context from both left and right of a token.
- Decoder-only LLMs (e.g., Mistral, Llama3): These models are pre-trained using autoregressive next-token prediction, where representations are conditioned on preceding tokens.
- Forecasting Module: They investigate two methods for integrating LLM-generated token-level representations into the forecasting module:
- Bottleneck Representation: This involves appending an End-of-Sequence (EOS) token to the input and using its final representation to predict the return. This method aims to compress the sequence information into a single vector.
- Aggregated Representation: This method involves averaging the token-level representations. This approach is simpler and avoids potential issues with representation collapse in the last token.
The LLMs are fine-tuned jointly with the forecasting module to minimize the mean squared error between predicted and actual returns.
Datasets
The study utilizes company-level financial newsflow data from 2003 to 2019, provided by a financial data vendor. Each news item is associated with company identifiers. The research employs three investment universe datasets: North American (NA), European (EU), and Emerging Markets (EM). These datasets include dates, stock identifiers, and true monthly forward returns. The data is split into training (2003-2014) and out-of-sample testing periods.
Key Insights and Ideas
- Aggregated Representations Shine: The aggregated representation method generally leads to better portfolio performance than the bottleneck approach, suggesting that averaging token embeddings is a robust way to capture predictive signals.
- Decoder-Only LLMs Excel in Larger Markets: In larger investment universes (like North America), decoder-only LLMs (Mistral, Llama3) tend to produce stronger portfolios compared to encoder-only LLMs.
- LLMs Outperform Sentiment Analysis: LLM-derived return predictions serve as a strong signal for portfolio construction, consistently outperforming conventional sentiment scores (from FinBERT and FinVADER) in terms of returns and Sharpe ratios.
- News Topics Matter Differently: Qualitative analysis reveals that while earnings-related events are important for both high and low return predictions, other topics like clinical trials and antitrust matters are more varied for low return predictions. Different LLMs also show varying focuses on news topics, suggesting potential for future research in aligning these focuses.
Key Results
- Portfolio Performance: LLM-based prediction portfolios generally yield higher returns and Sharpe ratios than the universe average and sentiment-based portfolios. For instance, in the North American universe, LLM-based portfolios achieved annual returns up to 27.00% with a Sharpe Ratio of 1.32 (Llama Bottleneck), significantly outperforming sentiment-based portfolios.
- Representation Method Impact: The aggregated representation method consistently outperformed the bottleneck method across most LLMs and universes, particularly in the EU and EM markets.
- LLM Architecture Impact: In the North American universe, decoder LLMs (Mistral, Llama3) generally led to stronger portfolios. However, in smaller universes, there were no consistent winners between encoder and decoder models.
- Long-Short Portfolio Nuances: For long-short portfolios, effective stock selection on both the long and short sides is crucial. Decoder-only LLMs showed a better ability to manage the short side, leading to smoother return curves and improved performance compared to encoder-only LLMs in some cases.
Limitations and Future Work
- Model Size and Universe Size: The paper notes that the underperformance of the encoder-only DeBERTa in the large North American universe might be due to model size or other factors, and questions why DeBERTa shows varying performance in different small universes.
- Future Research Directions: The authors suggest evaluating recently proposed larger encoder-only LLMs and exploring the underlying mechanisms of LLM focus differences to achieve more consistent predictions across different models. Further research could also investigate aligning LLMs' focuses for more structured predictions.
Summary updated