Abstract
Generative models offer a direct way to model complex data. Among them, energy-based models provide us with a neural network model that aims to accurately reproduce all statistical correlations observed in the data at the level of the Boltzmann weight of the model. However, one challenge is to understand the physical interpretation of such models. In this study, we propose a simple solution by implementing a direct mapping between the energy function of the Restricted Boltzmann Machine and an effective Ising spin Hamiltonian that includes high-order interactions between spins. This mapping includes interactions of all possible orders, going beyond the conventional pairwise interactions typically considered in the inverse Ising approach, and allowing the description of complex datasets. Earlier works attempted to achieve this goal, but the proposed mappings did not do properly treat the complexity of the problem or did not contain direct prescriptions for practical application. To validate our method, we performed several controlled numerical experiments where we trained the RBMs using equilibrium samples of predefined models containing local external fields, two-body and three-body interactions in various low-dimensional topologies. The results demonstrate the effectiveness of our proposed approach in learning the correct interaction network and pave the way for its application in modeling interesting datasets. We also evaluate the quality of the inferred model based on different training methods.
Introduction
In statistical physics, the "inverse Ising problem" is a fundamental challenge: given observed configurations of a system (such as spin states), can we infer the underlying interaction strengths (couplings) that generated them? While traditional methods excel at identifying pairwise interactions, they often struggle with higher-order correlations—three-body or four-body interactions—that are prevalent in complex biological, social, and physical systems.
Restricted Boltzmann Machines (RBMs) are powerful generative models capable of capturing these high-order patterns. However, they are often treated as "black boxes" because their internal weights and biases lack a direct physical interpretation. This research bridges that gap by providing a mathematical framework to translate RBM parameters into the language of many-body physics, specifically a Generalized Ising Model (GIM).
Methodology: Mapping RBMs to Physical Hamiltonians
The core contribution of this work is an exact mathematical mapping between the RBM architecture and a physical energy function. An RBM consists of a visible layer (representing the data) and a hidden layer (capturing latent features), connected by a weight matrix.
The authors derive analytical expressions for external fields, pairwise couplings, and $n$-body interactions as functions of the RBM's weights ($W$) and biases ($b, c$). To ensure computational feasibility for large-scale systems, they employ a Gaussian approximation based on the Central Limit Theorem to estimate the expected values of these couplings.
Furthermore, the study investigates how training dynamics influence the physical validity of the model. They compare Persistent Contrastive Divergence (PCD), which seeks equilibrium, against Random Initialization (Rdm-K), an out-of-equilibrium approach. The findings suggest that for the inferred couplings to be physically meaningful, the RBM must be trained in a "quasi-equilibrium" state to avoid learning the dynamics of the sampling process rather than the true statistics of the data.
Experimental Setup and Datasets
To validate the mapping, the researchers used synthetic datasets with known "ground truth" parameters:
- 1D Ferromagnetic Ising Model: A basic chain with nearest-neighbor interactions.
- 2D Ferromagnetic Ising Model: A square lattice featuring complex connectivity and phase transitions.
- 2D Edwards-Anderson Model: A disordered spin glass with mixed positive and negative interactions.
- 1D Model with 3-Body Interactions: Specifically designed to test the recovery of higher-order terms.
Results and Interpretation
The numerical tests followed a four-step pipeline: generating equilibrium samples, training the RBM, inferring effective couplings, and comparing them against the true values.
The results demonstrate that the RBM-based approach successfully recovers the interaction topology across all models (1D, 2D, and disordered). Key findings include:
- High-Order Recovery: Unlike traditional inverse Ising methods, this framework successfully identifies 3-body interactions.
- Statistical Efficiency: Inference error scales as $1/\sqrt{M}$ (where $M$ is the sample size), and the method outperforms traditional techniques like Belief Propagation in low-sample regimes.
- Capacity Requirements: Capturing the full interaction network requires a minimum number of hidden units ($N_h$), typically scaling with the number of visible units ($N_v$).
Conclusion
This work transforms the RBM from a black-box generative model into an interpretable inference engine for many-body systems. By providing a direct mapping to the Generalized Ising Model, it allows researchers to extract physical insights from complex datasets.
While the current Gaussian approximation is robust, future improvements may involve Large Deviation Theory to handle "heavy-tailed" weight distributions. Additionally, addressing the challenges of training RBMs near phase transitions (where dynamics slow down) remains a priority for applying this method to real-world datasets in fields like neuroscience and protein folding.
Summary updated