TL;DR
This paper introduces SAMBA, a novel framework for stock price prediction that combines the efficiency of the Mamba architecture with the relational modeling power of Graph Neural Networks (GNNs). SAMBA achieves state-of-the-art prediction accuracy with significantly reduced computational complexity compared to Transformer models, making it more suitable for real-time financial forecasting.
What Problem Does This Paper Address?
Accurately predicting stock market movements is crucial for financial gains, but the inherent complexity and volatility of markets make it a challenging task. While deep learning models like LSTMs and CNNs have been used, Transformer models have shown superior performance by capturing long-range dependencies. However, Transformers suffer from high computational and memory costs, limiting their use in real-time trading scenarios, especially with long historical data sequences. This paper aims to bridge this gap by developing a model that is both accurate and computationally efficient.
Methods
SAMBA is built upon two core components:
- Bidirectional Mamba (BI-Mamba) Block: This component leverages the Mamba architecture, which is known for its near-linear computational complexity and its ability to efficiently process long sequences using a selective scan mechanism. By making the Mamba block bidirectional, SAMBA can capture long-term dependencies in historical price data more effectively than a unidirectional Mamba model.
- Adaptive Graph Convolutional (AGC) Block: This block addresses the interdependencies between different daily stock features. It models these features as nodes in a graph and uses an adaptive graph convolution to learn the relationships between them. Crucially, the graph structure is learned end-to-end, adapting to the specific prediction task rather than relying on pre-defined similarity measures. This allows SAMBA to capture both temporal and relational patterns in the data.
The overall SAMBA model integrates these blocks, first using BI-Mamba to process sequential price data and then employing AGC to model feature interactions.
Datasets
The paper evaluates SAMBA on three real-world US stock market datasets:
- NASDAQ
- New York Stock Exchange (NYSE)
- Dow Jones Industrial Average (DJIA)
These datasets consist of 82 daily stock features and cover the period from January 2010 to November 2023. The data is split into 80% for training, 5% for validation, and 15% for testing.
Key Insights and Ideas
- Bridging Efficiency and Accuracy: The core insight is that combining the sequence modeling strengths of Mamba with the relational modeling capabilities of GNNs can lead to a model that is both highly accurate and computationally efficient, overcoming the limitations of existing Transformer-based approaches.
- Bidirectionality for Enhanced Temporal Capture: Simply using Mamba is good, but making it bidirectional significantly improves its ability to capture long-term historical dependencies in stock prices.
- Adaptive Graph Construction for Feature Relationships: Pre-defined graph structures can be suboptimal. The paper demonstrates the value of an end-to-end learned, adaptive graph structure that dynamically captures complex, non-linear relationships between stock features.
- Parameter Efficiency through Matrix Factorization: The AGC block's parameter count can be substantial. Using matrix factorization to generate graph convolution weights significantly reduces parameters, mitigating overfitting and improving generalization.
Key Results
SAMBA significantly outperforms several state-of-the-art baseline models (LSTM, Transformer, FreTS, StockMixer, AGCRN, FourierGNN, MambaStock) across all three datasets in terms of prediction accuracy, measured by RMSE, Information Coefficient (IC), and Rank Information Coefficient (RIC).
- Accuracy Improvements: SAMBA achieved notable RMSE improvements: 11.72% for NASDAQ, 10.07% for NYSE, and 6.90% for DJIA compared to the second-best models. IC and RIC improvements were also substantial.
- Computational Efficiency: While not the absolute fastest, SAMBA offers a favorable balance. It is more computationally complex than simple MLP models and MambaStock but significantly more efficient than Transformer and GNN-based models like AGCRN and FourierGNN. For instance, it requires 0.11 Million MACs, considerably less than the Transformer.
Limitations and Future Work
- Limitations: The paper acknowledges that SAMBA has higher computational complexity than simpler MLP-based models and MambaStock, though it argues this is a trade-off for superior accuracy. The adaptive graph construction, while powerful, could still be prone to overfitting on smaller datasets due to the large number of parameters involved before factorization.
- Future Work: The authors propose investigating multi-step joint stock price prediction, which would involve considering correlations between multiple stock markets simultaneously.
Summary updated