A New Computational Method of Input Selection for Stock Market Forecasting with Neural Networks

Wei Huang, Shouyang Wang, Lean Yu, Yukun Bao, Lin WangLecture Notes in Computer Science 2006 · DOI · OpenAlex

Introduction

Predicting stock market trends is a complex task characterized by noise, non-stationarity, and non-linear relationships. While Neural Networks (NNs) are powerful tools for pattern recognition, their effectiveness is heavily dependent on the quality of input variables. A common challenge is the "selection dilemma": while recent historical prices (lags) are highly correlated with future prices, they are also highly correlated with each other. This redundancy can lead to overfitting and diminished out-of-sample performance. This paper proposes a novel data-driven method to select a set of inputs that are both predictive and independent.

Methodology

The authors introduce a greedy search algorithm designed to optimize the lag structure of the input data. The core of this method is a selection criterion that balances individual predictive power against collective redundancy.

Flowchart of the proposed input selection algorithm

The algorithm follows these steps:

  1. Initialization: Define the maximum lag period ($N$) and the forecasting horizon ($n$).
  2. Selection Formula: Lags are selected by maximizing a ratio that penalizes redundant information: $$s_{i+1} = \arg \max_{s_i < j \le N} \left( \frac{|r_{j+n}|}{\prod_{k=1}^i |r_{j-s_k}|} \right)$$ The numerator represents the correlation between a potential lag and the target future value, while the denominator is the product of correlations between the potential lag and previously selected lags.

  3. Iterative Search: The process repeats until the search space is exhausted, ensuring each new input adds unique information.

Experimental Setup

The method was evaluated using daily observations from the S&P 500 and NIKKEI 225 indices between January 2001 and November 2005. Data from 2001 to 2004 was used for training, with 2005 reserved for out-of-sample testing. The selection method was applied to both Back-Propagation Networks (BPN) and Radial Basis Function Networks (RBFN).

Key Results

Performance was evaluated using Normalized Mean Squared Error (NMSE), where lower values indicate higher accuracy. The proposed method significantly outperformed traditional selection criteria.

Comparison of NMSE performance across different selection methods

As shown in the results, the proposed method achieved an NMSE of approximately 0.09, consistently beating benchmarks:

  • Naïve Prediction: NMSE ranged from 0.21 to 0.27.
  • AIC/BIC: While reasonable, these methods were less consistent than the proposed approach.
  • Chaos Analysis Criterion (CAC): Performed poorly (NMSE 0.32–0.38), often failing to beat the naïve benchmark because it selected highly redundant, adjacent lags.

The findings suggest that "information diversity" is more critical for neural network performance than raw correlation alone. By selecting lags that provide unique perspectives, the model avoids the pitfalls of multi-collinearity.

Conclusion and Future Work

The proposed ratio-based selection method offers a robust, data-driven approach to input selection for stock market forecasting. It is model-independent and stable across different lag settings. However, future research could improve the method by incorporating non-linear dependency measures like Mutual Information, as the current approach relies on linear autocorrelation. Additionally, exploring dynamic selection methods that adapt to changing market regimes could further enhance predictive stability.

Summary updated

BibTeX
@inproceedings{huang2006new,
  author = {Wei Huang and Shouyang Wang and Lean Yu and Yukun Bao and Lin Wang},
  title = {{A New Computational Method of Input Selection for Stock Market Forecasting with Neural Networks}},
  booktitle = {Lecture Notes in Computer Science},
  year = {2006},
  doi = {10.1007/11758549\_46},
}
Figure