Abstract: One major problem in applying neural networks to financial applications is the large number of features involved. The feature set is large because we simply do not know which of the given features may be useful to the system and include rather than risk throwing away a potentially useful one. In practice, training with the full set of features usually introduces unnecessary complexity and often degraded prediction performance. An important contribution of this paper is to focus the attention of neural network researchers on the need for a systematic feature preprocessing methodology for the purpose of improving predictability. The approach taken in this paper is to select subsets of the full feature sets that improve the prediction. We discuss two different feature subset selection algorithms: the Penalty selection algorithm and the feature elimination algorithm. We explain the criterion MisMatch we use to evaluate a feature set. Both of the proposed algorithms are described using this criterion. Improved accuracies are obtained with both of the subsets compared to using all the features on the DM-US exchange rate Tuesday return prediction task.
Introduction: The Curse of Dimensionality in Finance
When applying neural networks to financial forecasting, researchers often encounter the "curse of dimensionality." Because it is rarely clear which economic indicators—such as volatilities, trends, or cross-currency rates—actually drive market movements, the standard practice is to include as many features as possible.
However, this "feature bloat" leads to two significant problems. First, larger networks are computationally expensive and prone to overfitting. Second, the inclusion of "malicious" features can actively degrade performance. The authors define maliciousness as a state where two input vectors are similar in feature space but correspond to contradictory outcomes (e.g., one leads to a market surge, the other to a crash). These samples confuse the learning algorithm, making it difficult to extract a clear signal from the noise.
The SupNet Architecture
To address these issues, the researchers utilize a specific architecture called SupNet (Supervised Clustering Network). Unlike standard backpropagation models, SupNet is a two-layer network designed to group input vectors into clusters based on their target values (the desired prediction).
In this architecture, the input layer feeds into a cluster layer where nodes represent specific market conditions. This structure allows the model to evaluate how well a specific subset of features can separate different market outcomes without the "confusion" caused by irrelevant data.
Systematic Feature Selection Algorithms
The paper introduces two primary methodologies for pruning feature sets, both centered on a metric called the MisMatch Criterion. This criterion evaluates how much a specific subset of features confuses the clustering logic of the SupNet.
- The Penalty Selection Algorithm: This method assigns a "penalty" score to each individual feature based on the number of mispredictions it causes. Features that consistently lead to "MisMatches"—where the model predicts an uptrend but the reality is a downtrend, or vice versa—are discarded.
- The Feature Elimination Algorithm: This is a more computationally intensive recursive approach. It removes one feature at a time, calculates the resulting MisMatch count for the remaining subset, and retains the configuration that minimizes error. This process repeats until only the most essential features remain.
Experimental Results: DM-US Exchange Rate
The algorithms were tested on the DM-US Exchange Rate (German Mark vs. U.S. Dollar) using daily closing bids from 1973 to 1987. The initial feature set consisted of 71 indicators, including past daily returns, volatilities, and tangent slope indicators for five major currencies.
Key Insights
- The Canadian Dollar Paradox: Interestingly, features associated with the Canadian Dollar were almost always eliminated. The authors suggest that because the Canadian and U.S. economies are so tightly coupled, the CAD/USD rate provides no independent predictive power for the German Mark.
- Quality over Quantity: Using the full 71-feature vector resulted in performance no better than a coin flip. However, when pruned down to just 11–14 key indicators, the model's accuracy for "Uptrend" predictions jumped from 47% to 65%.
- Error Reduction: In multiple-cluster experiments, the "critical misprediction count" (predicting an uptrend during a downtrend) dropped from 10 using all features to zero using the eliminated feature set.
Conclusion
This research demonstrates that in financial forecasting, more data is not always better. By systematically removing irrelevant or "malicious" features, neural networks can achieve significantly higher predictive accuracy with lower computational complexity. While the Feature Elimination algorithm is computationally expensive, the resulting performance gains suggest that feature preprocessing is a vital step in developing robust financial trading models.
Summary updated