Abstract
The mean-field (MF) approximation provides a computationally efficient method for inferring interactions in networks of correlated variables. However, its performance relies heavily on strong regularization, a requirement often misunderstood as a mere defense against sampling noise. This study demonstrates that large regularization—specifically pseudocounts and $L_2$-norm penalties—is fundamentally necessary to correct systematic biases inherent in the MF approximation itself. Even in the limit of infinite data, a finite regularization strength is required to achieve optimal inference.
The Systematic Bias of Mean-Field Inference
Inferring the underlying interactions ($J$) from observed correlations is the core of the "Inverse Ising" and "Inverse Potts" problems. While the Mean-Field (MF) approximation is favored for its speed, it assumes variables follow a Gaussian distribution. Because binary spins (Ising) or categorical states (Potts) are non-Gaussian, the MF approach systematically overestimates strong couplings.
As shown in the figure above, the inferred coupling $J^{MF}$ deviates significantly from the true coupling $J$ as the interaction strength increases. This mathematical flaw persists regardless of the amount of data collected, necessitating a permanent "correction factor" in the form of regularization.
Regularization as a Necessary Correction
Standard statistical theory suggests that regularization should vanish as the number of samples ($B$) increases. However, in MF inference, the optimal regularization strength remains finite even as $B \to \infty$. The authors explore two primary schemes:
- Pseudocounts: Adding a uniform "fake" background to the observed data.
- $L_2$-norm penalties: Penalizing the magnitude of the interaction weights.
Through analytical study of the $m$-component spin model and numerical simulations of 1D lattices and random graphs, the researchers found that pseudocounts are particularly robust. They effectively "stretch" the inference where it is most needed, outperforming $L_2$ penalties in heterogeneous networks where node activity varies widely.
Key Results and Performance
The study reveals that for Ising models, a pseudocount intensity of $\alpha \approx 0.2$ is consistently optimal. In Potts models, which are significantly harder to infer, the required strength increases with the number of states ($q$), sometimes reaching $\alpha \approx 0.75$ for $q=20$.
The error analysis demonstrates that without regularization ($\alpha=0$), the relative square error for strong couplings ($J=2$) remains high even with many samples. Applying an optimal pseudocount ($\alpha=0.2$) drastically reduces this error and stabilizes the inference against sampling noise. This stability is crucial for practical applications, such as predicting protein-protein contacts, where the relative rank of interactions is more important than their absolute values.
Conclusion
The necessity of "abnormally strong" regularization in mean-field inference is not a failure of data collection but a requirement for correcting the approximation's intrinsic Gaussian bias. Pseudocounts offer a superior regularization strategy for complex, heterogeneous systems. While the Ising model is well-handled by these methods, Potts models remain challenging, particularly for highly conserved or extremely rare states, suggesting a need for future development of site-specific or link-dependent penalties.
Summary updated