Enhanced Financial Sentiment Analysis and Trading Strategy Development Using Large Language Models

Kemal Kirtac, Guido GermanoACL 2024

TL;DR

This paper introduces a novel approach to financial sentiment analysis and trading strategy development by leveraging large language models (LLMs). The study demonstrates that LLMs, particularly the OPT model, significantly outperform traditional methods in predicting stock market returns, leading to more profitable trading strategies.

What Problem Does This Paper Address?

Financial markets are notoriously complex, and predicting stock returns has long been a challenging endeavor. Traditional methods for analyzing financial news often rely on rule-based dictionaries or simpler machine learning models, which struggle to capture the nuanced language and evolving sentiment present in financial texts. This gap limits the effectiveness of sentiment analysis in driving profitable trading strategies. The authors aim to bridge this gap by exploring how advanced LLMs can extract deeper insights from financial news to improve both sentiment analysis accuracy and the development of sophisticated trading strategies.

Methods

The core of this research lies in fine-tuning pre-trained LLMs for financial sentiment analysis. The authors experimented with several LLMs, including OPT, BERT, FinBERT, LLAMA 3, and RoBERTa, adapting them to the specific domain of financial news. A key methodological innovation is the definition of sentiment labels based on an aggregated three-day excess stock return following a news article's publication. This approach captures a more comprehensive market reaction than a single-day outlook.

For sentiment analysis, the fine-tuned LLMs generate sentiment scores for news articles. These scores are then used in two primary ways:

  1. Predictive Analysis: A linear regression model is employed to assess the correlation between LLM-generated sentiment scores and subsequent stock returns, controlling for firm and time fixed effects.
  2. Trading Strategy Development: The sentiment scores are used to construct various portfolios (long, short, and long-short) based on the top 20% of positive and negative sentiment stocks. These strategies are then evaluated against benchmark portfolios.

Datasets

The study utilizes two primary datasets:

  • CRSP (Center for Research in Security Prices): Provides daily stock returns for U.S. companies.
  • Refinitiv: Supplies a comprehensive collection of U.S. financial news articles and quick alerts.

The researchers processed a large dataset of 965,375 U.S. financial news articles from January 1, 2010, to June 30, 2023. After applying filters for data quality and redundancy (e.g., novelty score based on cosine similarity), the final dataset for analysis comprised 965,375 unique news articles. This dataset was split with 20% for testing, and an additional 20% randomly selected from the remaining data for validation, leaving a training set of 193,070 articles.

Key Insights and Ideas

  • LLMs Excel at Nuance: The study strongly suggests that LLMs, due to their deep learning architecture, are far superior to traditional dictionary-based methods in understanding the subtle and evolving language of financial news.
  • Three-Day Return Window is Crucial: Defining sentiment based on a three-day aggregated excess return provides a more robust signal for predicting market movements than a single-day outlook, reflecting a more realistic market reaction.
  • OPT Leads the Pack: Among the LLMs tested, the OPT model demonstrated the highest accuracy in sentiment analysis and contributed to the most profitable trading strategies, highlighting the impact of model architecture and scale.
  • Beyond Simple Sentiment: The research moves beyond basic positive/negative sentiment to demonstrate how LLM-generated scores can directly inform complex portfolio construction and risk management.

Key Results

  • Sentiment Analysis Accuracy: The OPT model achieved the highest prediction accuracy of 74.4% for stock market returns, outperforming other LLMs and significantly exceeding the performance of the Loughran-McDonald dictionary model (50.1% accuracy).
  • Trading Strategy Performance: A self-financing trading strategy based on OPT scores achieved a Sharpe ratio of 3.05, substantially higher than the 1.23 Sharpe ratio achieved by a strategy using the Loughran-McDonald dictionary.
  • Predictive Power: Regression analysis showed that LLM sentiment scores, particularly from OPT, have a significant positive correlation with next-day stock returns, indicating strong predictive capabilities.

Limitations and Future Work

  • Computational Cost: Fine-tuning LLMs is computationally intensive, requiring significant resources (GPUs, TPUs) and time, which may limit accessibility for smaller organizations.
  • Overfitting Risk: Specialized LLMs like FinBERT face a higher risk of overfitting to financial texts, potentially limiting their generalization in rapidly changing markets. Conversely, broader models might introduce noise.
  • Data Scope: The study's reliance on U.S. financial news may not fully capture global financial trends. The quality and reliability of news sources can also impact accuracy.
  • Evaluation Metrics: While robust, the metrics used (accuracy, precision, recall, Sharpe ratio) may not fully capture all aspects of real-world trading performance, which is influenced by dynamic market conditions and investor behavior.

Future work suggested includes exploring more efficient training algorithms and model compression techniques, investigating LLMs in diverse global financial contexts, and addressing regulatory and ethical questions surrounding AI in finance. The authors also suggest exploring active trading strategies and determining optimal parameters for trading algorithms.

Summary updated

BibTeX
@inproceedings{kirtac2024enhanced,
  author = {Kemal Kirtac and Guido Germano},
  title = {{Enhanced Financial Sentiment Analysis and Trading Strategy Development Using Large Language Models}},
  booktitle = {Annual Meeting of the Association for Computational Linguistics},
  year = {2024},
}
Figure