A new cascaded projection pursuit network for nonlinear regression

Shih-Shien You, Jenq–Neng Hwang, I‐Chang Jou, Shyh-Rong Lay

Abstract

Cascaded correlation is a popular supervised learning architecture that dynamically grows layers of hidden neurons of fixed nonlinear activations (e.g., sigmoids), so that the network topology (size, depth) can be efficiently determined. Similar to a cascaded correlation learning network (CCLN), a projection pursuit learning network (PPLN) also dynamically grows the hidden neurons. Unlike a CCLN where cascaded connections from the existing hidden units to the new candidate hidden unit are required to establish high-order nonlinearity in approximating the residual error, a PPLN approximates the high-order nonlinearity by using trainable nonlinear nodal activation functions (e.g., hermite polynomials). To relax the necessity of predefined smoothness of nonlinearity (e.g., the order of hermite polynomials) in a PPLN, the authors propose a new learning network, called a cascaded projection pursuit network (CPPN), which combines both advantages of a PPLN and a CCLN. The training strategy of a CPPN is similar to that of PPLN and the added cascaded connections allow the CPNN to better capture the high order features without the necessity of proper selection of orders of polynomials. Simulation results show that a CPPN is suitable for nonlinear regression and outperforms a PPLN when the polynomial order used in the activation functions is less than the order of target functions.< <ETX xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">&gt;</ETX>
BibTeX
@inproceedings{you1994new,
  author = {Shih-Shien You and Jenq–Neng Hwang and I‐Chang Jou and Shyh-Rong Lay},
  title = {{A new cascaded projection pursuit network for nonlinear regression}},
  booktitle = {IEEE International Conference on Acoustics, Speech and Signal Processing},
  year = {1994},
  doi = {10.1109/icassp.1994.389588},
}

Loading related content...

Figure