Abstract
Gibbs measures, including Ising models and Exponential-family Random Graph Models (ERGMs), are foundational tools for modeling high-dimensional discrete data. However, these models frequently suffer from "model degeneracy," a phenomenon where the probability mass concentrates on extreme, unrealistic configurations, making statistical inference and simulation nearly impossible. This paper introduces Tapered Gibbs Measures, a principled extension of the maximum entropy framework that incorporates a quadratic penalty on extreme statistics. We prove that this approach can eliminate phase transitions, ensuring a stable "uniphase" distribution that allows for the successful estimation of complex models previously deemed unfittable.
The Challenge of Model Degeneracy
Gibbs measures are widely used in fields ranging from computer vision to social network analysis. They are typically derived using the principle of maximum entropy: finding the distribution that is the least informative while matching certain observed mean statistics.
Despite their utility, these models have a notorious "dark side" known as phase transitions or model degeneracy. In a social network model, for instance, a infinitesimal change in a parameter can cause the model to jump abruptly from predicting a sparse, realistic network to a completely "saturated" one where every possible connection exists. For researchers, this is a significant hurdle: Markov Chain Monte Carlo (MCMC) sampling often fails as the algorithm becomes trapped in these extreme modes, and the resulting model may place almost all its probability mass on configurations that bear no resemblance to the actual data.
Methodology: Tapered Gibbs Measures
The authors propose the Tapered Gibbs Measure to address this instability. While standard Gibbs measures maximize entropy subject to a constraint on the mean of sufficient statistics, the tapered version adds a second constraint: the variance of these statistics must also be bounded.
Using Karush-Kuhn-Tucker (KKT) multipliers, the resulting probability mass function includes an additional quadratic "tapering" term:
$$q(x|\theta, \beta) \propto \exp\left(\sum \theta_k g_k(x) - \sum \beta_k^{-2} (\mu_k - g_k(x))^2\right)$$
In this formulation, $\beta$ acts as a hyper-parameter controlling the strength of the force pulling the distribution toward the observed mean.
The Uniphase Guarantee
A key theoretical contribution of this work is the "Uniphase Guarantee" (Theorem 2). The authors provide a mathematical proof that for a sufficiently small $\beta$, the distribution is guaranteed to be unimodal, effectively removing local minima or saddle points in the configuration density. This ensures that the model remains stable and computationally tractable.
Key Results and Applications
Stabilizing the Ising Model
To demonstrate the effect, the authors applied the tapered measure to a classic Ising model representing political affiliation (simulated on an $n \times n$ toroidal lattice). In the standard model, the distribution was bimodal, predicting a community that was either "all liberal" or "all conservative," despite the underlying data being a 50/50 split. As shown in Figure 2, the Tapered Gibbs Measure successfully collapsed these extreme modes into a single, realistic Gaussian-like distribution centered at the observed data.
Fitting the "Unfittable": Network Science Collaboration
The approach was also tested on a real-world dataset of co-authorship among 1,589 scientists. This network is notoriously difficult for traditional ERGMs due to its size and structural complexity. Previous attempts to fit a 12-parameter ERGM to this data using standard Gibbs measures failed as the MCMC process diverged.
Using the tapered model, the authors successfully achieved convergence. The resulting model provided meaningful insights into network transitivity and degree distributions, matching the observed data with high precision.
Conclusion and Practical Heuristics
Tapered Gibbs Measures represent a significant advancement in the stability of high-dimensional discrete models. While choosing the optimal $\beta$ remains somewhat domain-dependent, the authors suggest a practical "2-Sigma" rule: setting $\beta$ to twice the square root of the expected variance ($r=2$) consistently yields stable results across various applications. By constraining variance, researchers can now fit complex, informative models that were previously impossible to estimate due to the "beast" of phase transitions.
Summary updated