Optimal learning in multilayer neural networks

Ole Winther, B. Lautrup, Jiawen ZhangPhysical 1997 · DOI

Abstract

The generalization performance of two learning algorithms, Bayes algorithm and the ``optimal learning'' algorithm, on two classification tasks is studied theoretically. In the first example the task is defined by a restricted two-layer network, a committee machine, and in the second the task is defined by the so-called prototype problem. The architecture of the learning machine, in both cases, is defined to be a committee machine. For both tasks the optimal learning algorithm, which is optimal when the solution is restricted to a specific architecture, performs worse than the overall optimal Bayes algorithm. However, both algorithms perform better than the conventional stochastic Gibbs algorithm, especially for the prototype problem in which the task and the learning machine are very different.
BibTeX
@inproceedings{winther1997optimal,
  author = {Ole Winther and B. Lautrup and Jiawen Zhang},
  title = {{Optimal learning in multilayer neural networks}},
  booktitle = {Physical},
  year = {1997},
  doi = {10.1103/physreve.55.836},
}

Loading related content...

Figure