Can threshold networks be trained directly?
Lists
Only you see this
Abstract
Neural networks with threshold activation functions are highly desirable because of the ease of hardware implementation. However, the popular gradient-based learning algorithms cannot be directly used to train these networks as the threshold functions are nondifferentiable. Methods available in the literature mainly focus on approximating the threshold activation functions by using sigmoid functions. In this paper, we show theoretically that the recently developed extreme learning machine (ELM) algorithm can be used to train the neural networks with threshold functions directly instead of approximating them with sigmoid functions. Experimental results based on real-world benchmark regression problems demonstrate that the generalization performance obtained by ELM is better than other algorithms used in threshold networks. Also, the ELM method does not need control variables (manually tuned parameters) and is much faster.
BibTeX
@article{huang2006can,
author = {Guang-Bin Huang and Qin-Yu Zhu and Kezhi Mao and Chee Kheong Siew and Paramasivan Saratchandran and Narasimhan Sundararajan},
title = {{Can threshold networks be trained directly?}},
journal = {IEEE Trans. Circuits Syst. II Express Briefs},
year = {2006},
url = {https://openreview.net/forum?id=rxaoKtcVIl},
}Loading related content...