Mean-field theory of Boltzmann machine learning

Toshiyuki TanakaPhysical 1998 · DOI

Abstract

I present a mean-field theory for Boltzmann machine learning, derived by employing Thouless-Anderson-Palmer free energy formalism to a full extent. Using the Plefka expansion an extended theory that takes higher-order correction to mean-field free energy formalism into consideration is presented, from which the mean-field approximation of general orders, along with the linear response correction, are derived by truncating the Plefka expansion up to desired orders. A theoretical foundation for an effective trick of using ``diagonal weights,'' introduced by Kappen and Rodr\'{\i}guez, is also given. Because of the finite system size and a lack of scaling assumptions on interaction coefficients, the truncated free energy formalism cannot provide an exact description in the case of Boltzmann machines. Accuracies of mean-field approximations of several orders are compared by computer simulations.
BibTeX
@inproceedings{tanaka1998mean,
  author = {Toshiyuki Tanaka},
  title = {{Mean-field theory of Boltzmann machine learning}},
  booktitle = {Physical},
  year = {1998},
  doi = {10.1103/physreve.58.2302},
}

Loading related content...

Figure