Abstract
We introduce the chi-square test neural network ($\chi^2NN$): a single hidden layer backpropagation neural network using the chi-square test theorem to redefine the cost and error functions. While weights and thresholds are modified using the standard backpropagation algorithm, this approach ensures consistent data distribution across training and testing sets. Designed for binary classification, experimental results on real-world datasets indicate that the proposed algorithm significantly improves classification accuracy compared to traditional approaches.
Introduction
Standard Backpropagation Neural Networks (BPNNs) are fundamental tools in supervised learning. However, they frequently encounter two significant challenges: the tendency to get trapped in local minima and slow convergence rates for complex non-linear problems. These issues often arise from the traditional Mean Squared Error (MSE) cost function, which focuses on point-to-point error rather than the broader statistical distribution of the data. Furthermore, standard models do not guarantee consistent performance if the testing data distribution deviates from the training set. The $\chi^2NN$ addresses these gaps by integrating statistical distribution theory into the learning process.
Methodology
The $\chi^2NN$ architecture consists of a single hidden layer. While it retains the gradient descent mechanism for weight updates, it redefines the objective function to focus on distribution matching.
Architecture and Workflow
The network architecture for binary classification is designed to map inputs through a hidden layer to a single output node, as illustrated in Figure 1.
The training workflow involves several key steps:
- Feature Extraction: Principal Component Analysis (PCA) is employed to reduce raw data dimensions, mitigating noise and preventing the network from being overwhelmed.
- Input Space Partitioning: The input space is divided into $M$ mutually exclusive sections.
-
Redefined Cost Function: Instead of MSE, a chi-square-based error function is used: $$E = \frac{1}{2} \sum_{i=1}^{m} \left( \frac{v_i}{N} - p_i \right)^2$$ where $v_i$ represents the observed outputs in a section, and $p_i$ is the expected probability. This approach forces the network to minimize the discrepancy between the distribution of its predictions and the actual data distribution.
-
Statistical Stopping Criterion: The network stops training when the cost function falls below a critical value derived from the chi-square distribution table (with $M-1$ degrees of freedom), providing a mathematically grounded termination point.
Experimental Results
The performance of $\chi^2NN$ was evaluated against conventional BPNN and Support Vector Machines (SVM) using five datasets from the UCI Machine Learning Repository: Iris, ILPD (Indian Liver Patient Dataset), BA (Banknote Authentication), BCW (Breast Cancer Wisconsin), and Balloons.
As shown in the results, $\chi^2NN$ outperformed the standard BPNN in four out of five datasets. Notably, on the Balloons dataset, $\chi^2NN$ achieved 87.5% accuracy, significantly higher than the 75% achieved by both standard BPNN and SVM. On the high-stakes Breast Cancer Wisconsin (BCW) dataset, it reached 97.06% accuracy, surpassing the 94.12% of the standard BPNN. In several instances, such as the Iris and ILPD datasets, it matched or exceeded the performance of SVMs, which are typically considered more robust than single-layer networks.
Discussion and Conclusion
The core innovation of $\chi^2NN$ is treating neural network training as a distribution-matching problem. By ensuring that the proportion of samples in partitioned sections of the input space matches the expected distribution, the model becomes more robust to variations between training and testing sets.
While the results are promising, the current implementation is limited to binary classification. Future work will explore extending the chi-square cost function to multi-class problems. Additionally, the partitioning method faces the curse of dimensionality, as the number of sections grows exponentially with the number of features. Investigating the behavior of this statistical cost function in deeper architectures remains a key area for future research.
Summary updated