Abstract
Inferring pairwise interactions from data is a fundamental task in statistical mechanics, yet real-world datasets often contain higher-order correlations that bias standard pairwise models. We present a hybrid Energy-Based Model (EBM) that decomposes the system energy into an interpretable pairwise component and a flexible neural network component. This architecture allows the neural network to capture complex higher-order noise, enabling more accurate reconstruction of the underlying physical couplings. We demonstrate the effectiveness of this approach on synthetic Ising-like systems and protein sequence data, showing that the hybrid model consistently outperforms standard pairwise and pure neural network approaches in recovering true interaction strengths.
Introduction
In fields ranging from structural biology to condensed matter physics, researchers frequently employ pairwise models, such as the Ising or Potts models, to characterize the interactions between system components. The goal of inverse statistical mechanics is to infer these interaction strengths (couplings) from observed data. However, real-world systems often exhibit higher-order interactions involving three or more variables. When a simple pairwise model is fitted to such data, it often produces "effective" interactions that are distorted by these higher-order effects, obscuring the true underlying physics. This paper addresses the challenge of extracting clean pairwise information from data contaminated by complex, high-order noise.
Methods
The authors propose a hybrid Energy-Based Model (EBM) where the total energy of a configuration $s$ is defined as the sum of a pairwise term and a neural network term: $$E(s) = E_{pw}(s) + E_{nn}(s)$$ The term $E_{pw}$ contains the explicit coupling parameters $J_{ij}$ of interest, while $E_{nn}$ is a Multi-Layer Perceptron (MLP) designed to absorb higher-order interactions.
Training and Reconstruction
The model is trained using pseudolikelihood maximization, a technique that avoids the intractable partition function by predicting the value of a single variable conditioned on all others. This approach is analogous to the masking strategies used in modern transformer models. After training, the authors employ a Fourier expansion technique to extract the final "effective" pairwise interactions from the total energy function. This ensures that any pairwise patterns inadvertently learned by the neural network are correctly attributed to the final reconstructed couplings.
Results
The model was evaluated on synthetic datasets generated from Ising-like models with varying system sizes ($N=16, 64$) and varying strengths ($\gamma$) of higher-order interactions.
As shown in the performance plots, the hybrid model (both explicit and reconstructed versions) consistently achieves lower reconstruction error compared to "Pairwise Only" or "MLP Only" models. The advantage of the hybrid approach becomes more pronounced as the strength of higher-order interactions ($\gamma$) increases. Furthermore, the hybrid model demonstrated superior sample efficiency, requiring fewer data points to reach high accuracy than the pure MLP approach.
In real-world tests using the Pfam dataset for the WW domain protein, the hybrid model improved physical contact prediction by reducing false positives compared to standard Potts models. This suggests the model successfully filters out co-evolutionary noise that typically plagues traditional bioinformatics tools.
Discussion
The success of the hybrid model stems from a "division of labor": the neural network acts as a flexible sink for complex correlations, allowing the pairwise term to focus on the fundamental couplings. This architecture bridges the gap between the interpretability of simple physical models and the expressive power of deep learning. While the model faces challenges with specific "parity-like" interactions and computational scaling for very large systems, it represents a significant step toward more robust inverse statistical mechanics. Future work will focus on scaling these methods to larger protein families and exploring more complex interaction topologies.
Summary updated