Spline-based neuro-fuzzy Kolmogorov's network for time series prediction

Vitaliy KolodyazhniyEsann 2009

Abstract

Predicting chaotic time series remains a significant challenge due to the sensitivity of such systems to initial conditions and approximation errors. This paper introduces the Spline-based Neuro-Fuzzy Kolmogorov’s Network (SNFKN), which enhances the traditional Neuro-Fuzzy Kolmogorov’s Network (NFKN) by employing cubic B-splines as membership functions. This modification provides higher-order smoothness and improved non-linear approximation capabilities. Experimental results on the Mackey-Glass and Santa-Fe Laser benchmarks demonstrate a significant reduction in prediction error, outperforming existing neuro-fuzzy models by a factor of three to four while maintaining a parsimonious architecture.

Introduction

The prediction of chaotic time series is often hindered by the "curse of dimensionality." Kolmogorov’s Superposition Theorem (KST) offers a theoretical framework to decompose complex multi-variable functions into sums of univariate functions, providing a basis for efficient neural network architectures. While previous implementations like the Neuro-Fuzzy Kolmogorov’s Network (NFKN) utilized triangular membership functions for simplicity, these models often lack the smoothness required for high-precision forecasting in highly non-linear domains.

Methodology

The proposed SNFKN architecture introduces several key innovations to improve upon the NFKN framework:

B-Spline Membership Functions

The core advancement is the replacement of piecewise-linear triangular functions with cubic B-splines ($q=4$). These splines ensure $C^2$ continuity, allowing the network to capture the underlying curvature of chaotic systems more effectively than the "cornered" approximations of triangular functions.

Cubic B-spline membership functions providing smooth transitions for the SNFKN model.

Hybrid Learning and Initialization

To ensure robust and efficient training, SNFKN employs a hybrid learning rule. Linear Least Squares (LS) is used to analytically determine optimal weights for the output layer, while Gradient Descent fine-tunes the non-linear synapses in the hidden layer. Furthermore, the model utilizes Principal Component Analysis (PCA) on the membership degrees of training data for deterministic weight initialization, avoiding the inconsistencies associated with random starts.

Experimental Results

The model was benchmarked against two standard chaotic time series: the Mackey-Glass (MG) equation and the Santa-Fe Laser data.

Mackey-Glass Prediction

On the MG dataset, the SNFKN demonstrated superior performance compared to the standard NFKN. The model achieved a significantly lower Normalized Mean Squared Error (NMSE) with fewer training epochs, as detailed in the comparative analysis below.

Performance comparison between SNFKN and NFKN on the Mackey-Glass dataset, highlighting the accuracy gains of the spline-based approach.

Santa-Fe Laser Data

The Laser dataset, characterized by complex intensity oscillations, further validated the model's efficacy. The SNFKN achieved an NMSE of 0.0320, nearly four times more accurate than the NFKN's 0.1194. Notably, this performance rivaled the winner of the original Santa-Fe competition, despite the SNFKN's simpler architecture compared to specialized FIR-synapse networks.

Conclusion

The integration of cubic B-splines into a Kolmogorov-based neuro-fuzzy network significantly enhances its ability to model complex, chaotic dynamics. By providing a smooth, parsimonious, and interpretable framework, the SNFKN represents a substantial step forward in time series forecasting. Future work will explore adaptive knot positioning and online learning variants for real-time applications.

Summary updated

BibTeX
@inproceedings{kolodyazhniy2009spline,
  author = {Vitaliy Kolodyazhniy},
  title = {{Spline-based neuro-fuzzy Kolmogorov's network for time series prediction}},
  booktitle = {Esann},
  year = {2009},
}
Figure