On the closure of the set of functions that can be realized by a given multilayer perceptron

Marco Gori, Franco Scarselli, Ah Chung TsoiIEEE Trans. Neural Networks 1998 · DOI · OpenReview

Abstract

Given a multilayer perceptron (MLP) with a fixed architecture, there are functions that can be approximated up to any degree of accuracy, without having to increase the number of the hidden nodes. Those functions belong to the closure F~ of the set F~ of the maps realizable by the MLP. In this paper, we give a list of maps with this property. In particular, it is proven that: 1) rational functions belongs to F~ for networks with inverse tangent activation function; and 2) products of polynomials and exponentials belongs to F~ for networks with sigmoid activation function. Moreover, for a restricted class of MLPs, we prove that the list is complete and give an analytic definition of F~.
BibTeX
@article{gori1998closure,
  author = {Marco Gori and Franco Scarselli and Ah Chung Tsoi},
  title = {{On the closure of the set of functions that can be realized by a given multilayer perceptron}},
  journal = {IEEE Trans. Neural Networks},
  year = {1998},
  doi = {10.1109/72.728354},
}

Loading related content...

Figure