Harnessing neural networks: A random matrix approach

Cosme Louart, Romain CouilletICASSP 2017 · OpenReview

Abstract

This article proposes an original approach to the performance understanding of large dimensional neural networks. In this preliminary study, we study a single hidden layer feed-forward network with random input connections (also called extreme learning machine) which performs a simple regression task. By means of a new random matrix result, we prove that, as the size and cardinality of the input data and the number of neurons grow large, the network performance is asymptotically deterministic. This entails a better comprehension of the effects of the hyper-parameters (activation function, number of neurons, etc.) under this simple setting, thereby paving the path to the harnessing of more involved structures.
BibTeX
@inproceedings{louart2017harnessing,
  author = {Cosme Louart and Romain Couillet},
  title = {{Harnessing neural networks: A random matrix approach}},
  booktitle = {IEEE International Conference on Acoustics, Speech and Signal Processing},
  year = {2017},
  url = {https://openreview.net/forum?id=zlzXaO8Uno},
}

Loading related content...

Figure