A Random Matrix and Concentration Inequalities Framework for Neural Networks Analysis

Cosme Louart, Romain CouilletICASSP 2018 · OpenReview

Abstract

This article provides a theoretical analysis of the asymptotic performance of a regression or classification task performed by a simple random neural network. This result is obtained by leveraging a new framework at the crossroads between random matrix theory and the concentration of measure theory. This approach is of utmost interest for neural network analysis at large in that it naturally dismisses the difficulty induced by the non-linear activation functions, so long that these are Lipschitz functions. As an application, we provide formulas for the limiting law of the random neural network output and compare them conclusively to those obtained practically on handwritten digits databases.
BibTeX
@inproceedings{louart2018random,
  author = {Cosme Louart and Romain Couillet},
  title = {{A Random Matrix and Concentration Inequalities Framework for Neural Networks Analysis}},
  booktitle = {IEEE International Conference on Acoustics, Speech and Signal Processing},
  year = {2018},
  url = {https://openreview.net/forum?id=6rXPjAcikA},
}

Loading related content...

Figure