Optimizing Convolutional Neural Network-Long Short-Term Memory Architecture with Additive Attention Mechanism for Stock Price Prediction

Bin Wu, Youfu Li, Geng-Shen FuRevista 2024 · DOI

TL;DR

Researchers have developed a hybrid deep learning model (CNN-LSTM-AAM) that significantly improves stock price prediction by combining spatial feature extraction, temporal sequence modeling, and an additive attention mechanism. By intelligently weighting the most relevant historical time steps, the model outperforms standard CNN and LSTM configurations across various Chinese stocks and market indices.

What Problem Does This Paper Address?

Predicting the stock market is notoriously difficult due to its non-linear, non-stationary, and "noisy" nature. Traditional econometric models (like ARMA or GARCH) often fail because they rely on strict assumptions of linearity. While deep learning models like Long Short-Term Memory (LSTM) networks have improved our ability to handle sequences, they still face two major hurdles in financial contexts:

  1. Noise Sensitivity: Financial data is full of outliers that can destabilize standard neural networks.
  2. Information Dilution: In long sequences, the relative importance of specific past events can get "lost," leading to lower accuracy when the model treats all past time steps with equal weight.

The authors aim to bridge this gap by creating a more robust architecture that can filter noise and focus on the most impactful data points in a time series.

Methods

The proposed CNN-LSTM-AAM model follows a sophisticated multi-stage pipeline:

  • Data Refinement (PCA): Before hitting the neural network, the raw data (including 23 technical indicators like Bollinger Bands and RSI) undergoes Principal Component Analysis. This reduces dimensionality and filters out redundant, highly correlated noise.
  • Feature Extraction (CNN): A Convolutional Neural Network layer is used to capture "local" features or spatial patterns within the technical indicators.
  • Sequence Modeling (LSTM): The output flows into an LSTM layer, which is designed to memorize long-term dependencies and the chronological flow of market data.
  • The Innovation (Additive Attention Mechanism - AAM): This is the core contribution. Unlike standard LSTMs that treat the entire hidden state sequence equally, the AAM adaptively computes weights for different time steps. It uses learnable weight matrices to decide which specific days in the look-back window are most predictive of tomorrow’s price.
  • Optimization: The model uses a two-step parameter tuning process involving experience-based settings followed by a random search to find the optimal hyperparameters (like learning rate and batch size).

Datasets

The study validates the model using 20 years of data (January 2001 to December 2021) from the Chinese stock market, sourced via the BaoStock interface. The data includes:

  • Individual Stocks: Vanke A (VA), Shanghai International Port Group (SIPG), and China Merchants Bank (CMB).
  • Market Indices: China Securities 500 (CSI 500), Shanghai Stock Exchange 50 (SSE 50), and the Growth Enterprise Index (GEI).

The researchers used an 80/10/10 split for training, validation, and testing, respectively.

Key Insights and Ideas

  • Attention as a Filter: The "aha moment" here is that not all history is equally relevant. By adding the AAM layer after the LSTM, the model can effectively "ignore" noisy days and "focus" on high-signal periods, which is crucial in volatile markets.
  • The Synergy of Hybridization: The paper demonstrates that while CNNs are great at patterns and LSTMs are great at sequences, they are better together. The CNN acts as a sophisticated pre-processor for the LSTM, and the Attention mechanism acts as a sophisticated post-processor.
  • PCA is Non-Negotiable: The authors emphasize that high correlation between technical indicators harms the learning ability of neural networks. Using PCA to ensure independent components (keeping a Cumulative Contribution Rate > 85%) was vital for model stability.

Key Results

The CNN-LSTM-AAM (referred to as "Model-3rd") consistently achieved the lowest error rates across all metrics (MSE, MAE, and MAPE):

  • Vanke A (VA) Stock: The Mean Absolute Percentage Error (MAPE) dropped from 12.80% (standard LSTM) to just 3.86% with the full hybrid model.
  • Index Prediction: For the SSE 50 index, the hybrid model achieved a remarkably low MAPE of 4.18%, significantly outperforming the standard CNN-LSTM (9.63%).
  • Stability: Visual analysis of the loss curves showed that the AAM-integrated model converged faster and exhibited much less "jitter" during the training process compared to its predecessors.

Limitations and Future Work

While the model shows high technical proficiency, the authors acknowledge several limitations:

  • Purely Quantitative: The model currently only considers daily trading data and technical indicators. It does not account for "soft" data like investor sentiment (news/social media), national policies, or broader economic cycles.
  • Parameter Sensitivity: The authors noted that the model's performance varies across different stocks, suggesting that the architecture requires bespoke hyperparameter tuning for every new asset it encounters.
  • Future Directions: The researchers plan to integrate sentiment analysis and macroeconomic factors into the feature set. They also suggest exploring the model's application in other high-stakes time-series fields like earthquake prediction or trajectory tracking.

Summary updated

Abstract

Stock price fluctuations reflect market expectations for the economic situation and company profits. Accurately predicting stock prices has become a hot topic in academia. With the rapid development of artificial intelligence, many researchers are starting to use machine learning algorithms to predict stock prices. In this paper, a new time series prediction model, the combination of the convolutional neural network and long shortterm memory neural network with additive attention mechanism (CNNLSTM-AAM), is proposed for stock price prediction. It can combine the advantages of the convolutional neural network (CNN), the long shortterm memory (LSTM) neural network, and the additive attention mechanism (AAM), and better capture nonlinear features of time series data. In the simulation analysis, we select sample data of three stocks (Vanke A, Shanghai International Port Group, and China Merchants Bank) and three stock price indexes (China Securities 500 Index, Shanghai Stock Exchange 50 Index, and Growth Enterprise Index) in the Chinese stock market for comparative analysis, and use mean square error (MSE), mean absolute error (MAE), and mean absolute percentage error (MAPE) as the evaluation indexes. The CNN-LSTM-AAM model has the best prediction ability relative to the CNN and CNN-LSTM models. In addition, we also find that the prediction ability of the CNN-LSTM-AAM model for different stock data sets is different under the existing parameter conditions. For a specific dataset, the parameter conditions of the CNN-LSTM-AAM model need to be further adjusted to achieve the best prediction effect. Based on the above findings, the CNN-LSTM-AAM model has better performance and higher accuracy, and can provide credible decision-making basis and research methods for investors, financial institutions, and regulators.
BibTeX
@inproceedings{wu2024optimizing,
  author = {Bin Wu and Youfu Li and Geng-Shen Fu},
  title = {{Optimizing Convolutional Neural Network-Long Short-Term Memory Architecture with Additive Attention Mechanism for Stock Price Prediction}},
  booktitle = {Revista},
  year = {2024},
  doi = {10.23967/j.rimni.2024.10.56790},
}
Figure