Introduction
Predicting stock market trends is a complex task characterized by noise, non-stationarity, and non-linear relationships. While Neural Networks (NNs) are powerful tools for pattern recognition, their effectiveness is heavily dependent on the quality of input variables. A common challenge is the "selection dilemma": while recent historical prices (lags) are highly correlated with future prices, they are also highly correlated with each other. This redundancy can lead to overfitting and diminished out-of-sample performance. This paper proposes a novel data-driven method to select a set of inputs that are both predictive and independent.
Methodology
The authors introduce a greedy search algorithm designed to optimize the lag structure of the input data. The core of this method is a selection criterion that balances individual predictive power against collective redundancy.
The algorithm follows these steps:
- Initialization: Define the maximum lag period ($N$) and the forecasting horizon ($n$).
-
Selection Formula: Lags are selected by maximizing a ratio that penalizes redundant information: $$s_{i+1} = \arg \max_{s_i < j \le N} \left( \frac{|r_{j+n}|}{\prod_{k=1}^i |r_{j-s_k}|} \right)$$ The numerator represents the correlation between a potential lag and the target future value, while the denominator is the product of correlations between the potential lag and previously selected lags.
-
Iterative Search: The process repeats until the search space is exhausted, ensuring each new input adds unique information.
Experimental Setup
The method was evaluated using daily observations from the S&P 500 and NIKKEI 225 indices between January 2001 and November 2005. Data from 2001 to 2004 was used for training, with 2005 reserved for out-of-sample testing. The selection method was applied to both Back-Propagation Networks (BPN) and Radial Basis Function Networks (RBFN).
Key Results
Performance was evaluated using Normalized Mean Squared Error (NMSE), where lower values indicate higher accuracy. The proposed method significantly outperformed traditional selection criteria.
As shown in the results, the proposed method achieved an NMSE of approximately 0.09, consistently beating benchmarks:
- Naïve Prediction: NMSE ranged from 0.21 to 0.27.
- AIC/BIC: While reasonable, these methods were less consistent than the proposed approach.
- Chaos Analysis Criterion (CAC): Performed poorly (NMSE 0.32–0.38), often failing to beat the naïve benchmark because it selected highly redundant, adjacent lags.
The findings suggest that "information diversity" is more critical for neural network performance than raw correlation alone. By selecting lags that provide unique perspectives, the model avoids the pitfalls of multi-collinearity.
Conclusion and Future Work
The proposed ratio-based selection method offers a robust, data-driven approach to input selection for stock market forecasting. It is model-independent and stable across different lag settings. However, future research could improve the method by incorporating non-linear dependency measures like Mutual Information, as the current approach relies on linear autocorrelation. Additionally, exploring dynamic selection methods that adapt to changing market regimes could further enhance predictive stability.
Summary updated