Stock Market Price Prediction: A Hybrid LSTM and Sequential Self-Attention based Approach

Karan Pardeshi, Sukhpal Singh Gill, Ahmed M. AbdelmoniemarXiv 2023 · DOI · OpenReview

TL;DR

This paper introduces a novel hybrid deep learning model, LSTM-SSAM, combining Long Short-Term Memory (LSTM) with a Sequential Self-Attention Mechanism for stock price prediction. The model demonstrates superior accuracy and reduced error compared to existing methods, offering a more robust approach to forecasting based on historical price trends.

What Problem Does This Paper Address?

Predicting stock market prices is a highly sought-after but challenging task. While external factors like geopolitical events significantly influence stock prices, this paper focuses on the aspect of forecasting based purely on historical price trends and seasonality. The core problem addressed is the difficulty in accurately capturing the complex, often noisy, and non-linear relationships within historical stock data to make reliable future price predictions. Existing methods often struggle with long-term dependencies and the inherent unpredictability of financial markets, leading to significant errors.

Methods

The proposed solution is a hybrid model named LSTM-SSAM (Long Short-Term Memory with Sequential Self-Attention Mechanism).

  • Long Short-Term Memory (LSTM): LSTMs are a type of Recurrent Neural Network (RNN) well-suited for time-series data. They are designed to overcome the vanishing gradient problem in traditional RNNs, allowing them to learn long-term dependencies in sequential data. The paper utilizes an LSTM layer with 50 units and sets return_sequences to True to capture all hidden states.
  • Sequential Self-Attention Mechanism: This mechanism allows the model to weigh the importance of different parts of the input sequence when making a prediction. Instead of processing data strictly sequentially, self-attention enables the model to "look back" at relevant past data points, regardless of their position in the sequence. This is crucial for identifying subtle patterns and relationships that might be missed by a purely sequential LSTM. The paper details a standard self-attention architecture involving Query (Q), Key (K), and Value (V) vectors, scaled dot-product attention, and a softmax function.
  • Hybrid Approach: The LSTM layer processes the sequential data, and its output is then fed into the self-attention mechanism. This combination aims to leverage LSTM's ability to handle temporal dependencies with the self-attention mechanism's capacity to focus on the most relevant historical information.
  • Other Algorithms for Comparison: The paper also evaluates standard LSTM (with 1 and 50 units), CNN-BiLSTM, LSTM-CNN, Facebook Prophet, and ARIMA models to benchmark the performance of their proposed LSTM-SSAM.

Datasets

The study uses historical stock data from Yahoo Finance for three Indian banks:

  • SBIN (State Bank of India): A 10-year dataset (May 2012 to May 2022) with 2484 observations.
  • HDFCBANK (HDFC Bank)
  • BANKBARODA (Bank of Baroda)

The primary focus for prediction is the "Adjusted Close" price, which accounts for corporate actions like dividends and stock splits, making it a more accurate indicator of stock value than the simple closing price.

Key Insights and Ideas

  • Self-Attention Enhances LSTM: The core insight is that augmenting an LSTM with a self-attention mechanism significantly improves its ability to capture complex patterns in stock price data. The attention mechanism helps the model focus on the most informative historical data points, leading to more accurate predictions.
  • Importance of "Adjusted Close": Using the "Adjusted Close" price as the target variable is crucial for accurate forecasting, as it provides a more realistic representation of the stock's true value over time.
  • Time Series Data is Noisy: The paper acknowledges the inherent noise and unpredictability in stock market data due to various factors, highlighting the need for sophisticated models that can filter out irrelevant information and identify underlying trends.
  • Trade-offs in Model Configuration: The authors emphasize the importance of careful hyperparameter tuning (e.g., batch size, number of epochs, optimizer choice) to balance model performance and efficiency.

Key Results

The LSTM-SSAM model consistently outperformed other benchmark algorithms across the tested datasets.

  • SBIN Dataset:
    • LSTM-SSAM achieved an RMSE of 8.17 and an R² score of 0.946, significantly outperforming the LSTM (Unit 50) model (RMSE: 10.80, R²: 0.905).
  • The forecast error percentage for the proposed model was generally below ~3%.
  • HDFCBANK and BANKBARODA Datasets:
    • The proposed model achieved the best accuracy, with R² scores of 90.74% for HDFCBANK and 96.08% for BANKBARODA.
  • These results were superior to even the LSTM (Unit 50) model, which itself performed better than other traditional methods.

Limitations and Future Work

  • Focus on Historical Data: The current model relies solely on historical price data. The authors acknowledge that real-world stock market volatility is also influenced by external factors like news, social media sentiment, and geopolitical events.
  • Limited to Banking Sector: The experiments were conducted on stocks from the banking sector.
  • Future Work:
    • Incorporate External Data: Integrate stock news and Twitter sentiment analysis to enhance model stability and accuracy, especially during major events.
  • Distributed/Federated Learning: Explore deploying the approach using distributed or federated learning paradigms for more dynamic and adaptive stock price prediction systems.
  • Broader Sector Analysis: Extend the analysis to a wider range of stock sectors.

Summary updated

Abstract

One of the most enticing research areas is the stock market, and projecting stock prices may help investors profit by making the best decisions at the correct time. Deep learning strategies have emerged as a critical technique in the field of the financial market. The stock market is impacted due to two aspects, one is the geo-political, social and global events on the bases of which the price trends could be affected. Meanwhile, the second aspect purely focuses on historical price trends and seasonality, allowing us to forecast stock prices. In this paper, our aim is to focus on the second aspect and build a model that predicts future prices with minimal errors. In order to provide better prediction results of stock price, we propose a new model named Long Short-Term Memory (LSTM) with Sequential Self-Attention Mechanism (LSTM-SSAM). Finally, we conduct extensive experiments on the three stock datasets: SBIN, HDFCBANK, and BANKBARODA. The experimental results prove the effectiveness and feasibility of the proposed model compared to existing models. The experimental findings demonstrate that the root-mean-squared error (RMSE), and R-square (R2) evaluation indicators are giving the best results.
BibTeX
@misc{pardeshi2023stock,
  author = {Karan Pardeshi and Sukhpal Singh Gill and Ahmed M. Abdelmoniem},
  title = {{Stock Market Price Prediction: A Hybrid LSTM and Sequential Self-Attention based Approach}},
  year = {2023},
  eprint = {2308.04419},
  archivePrefix = {arXiv},
  url = {https://arxiv.org/abs/2308.04419},
}
Figure