Abstract
In statistical mechanics, measuring the number of available states and their probabilities, and thus the system's entropy, enables the prediction of the macroscopic properties of a physical system at equilibrium. This predictive capacity hinges on the knowledge of the a priori probabilities of observing the states of the system, given by the Boltzmann distribution. Unfortunately, the successes of equilibrium statistical mechanics are hard to replicate out of equilibrium, where the a priori probabilities of observing states are in general not known, precluding the naïve application of usual tools. In the last decade, exciting developments have occurred that enable the direct numerical estimation of the entropy and density of states of athermal and non-equilibrium systems, thanks to significant methodological advances in the computation of the volume of high-dimensional basins of attraction. Here, we provide a detailed account of these methods, underscoring the challenges that lie in such estimations, recent progress on the matter, and promising directions for future work.
The Challenge of Non-Equilibrium Entropy
In classical equilibrium statistical mechanics, macroscopic properties like temperature or pressure are predicted by knowing the probability of microscopic states, typically via the Boltzmann distribution. However, many real-world systems—such as granular matter (sand piles) or glasses—exist in states far from equilibrium. In these regimes, the a priori probabilities of states are unknown, making the calculation of entropy—the fundamental measure of disorder and stability—notoriously difficult.
To find entropy, one theoretically needs to count every possible stable arrangement (microstate). In the vast, high-dimensional configuration spaces of these systems, direct counting is astronomically impossible. This paper addresses this "curse of dimensionality" by shifting the problem from counting to sampling.
The Geometric Curse of High Dimensions
High-dimensional geometry is profoundly counter-intuitive. As the number of dimensions $d$ increases, the volume of an object like a hypercube or a basin of attraction concentrates in increasingly thin, "tentacle-like" corners. These regions are so narrow they act like "Gabriel’s Horn"—possessing finite volume but extreme extension.
Standard Monte Carlo (MC) sampling techniques fail in these landscapes because they almost never encounter these critical corner regions. Consequently, traditional methods severely underestimate volumes in high dimensions, leading to a total breakdown in entropy estimation beyond approximately 30 dimensions.
Methodology: Sampling Basins of Attraction
The authors leverage the concept of Basins of Attraction: the set of all initial conditions that lead a system to settle into a specific stable state (a local energy minimum).
- The Sampling Identity: Instead of counting all basins ($\Omega$), they use the identity $\Omega = \mathcal{V} / \langle v \rangle$, where $\mathcal{V}$ is the total volume of configuration space and $\langle v \rangle$ is the average volume of a single basin. This reduces the problem to measuring the average size of a representative sample of basins.
- Biased Random Walks: To map the "tentacles" of a basin, the authors tether a virtual "walker" to the center of a basin using a harmonic spring. By varying the spring's stiffness ($k$), they force the walker to explore different regions—from the dense center to the sparse, distant corners.
- MBAR Stitching: Using the Multi-State Bennett Acceptance Ratio (MBAR), the authors mathematically stitch together samples from multiple walkers with different stiffnesses. This allows them to reconstruct the full "density of states" and accurately calculate the basin volume.
Validation and Key Results
The method was first validated against a "ground truth" benchmark: the hypercube. While naive Monte Carlo sampling fails to measure a hypercube's volume accurately beyond $d \approx 30$, the basin-volume method remains precise up to 1,000 dimensions.
Beyond mathematical benchmarks, the framework was applied to Granular Packings (simulations of 64 polydisperse disks). This led to a major physical insight: the death of Edwards' Hypothesis. For decades, physicists debated if all stable arrangements of a granular pile are equally likely. The authors demonstrated that this is only true at the exact point of unjamming. Above that density, the landscape is hierarchical, with low-energy states possessing significantly larger basins than high-energy states.
Future Directions
While computationally expensive—requiring numerous energy evaluations per step to ensure the walker stays within a specific basin—the potential applications are broad. The authors suggest that this framework could be used to study the "loss landscapes" of Deep Neural Networks. Measuring the volume of "flat minima" in AI models could finally provide a rigorous explanation for why certain architectures generalize better than others, turning a physical tool into a lens for understanding machine intelligence.
Summary updated