Abstract
In statistical mechanics, measuring the number of available states and their probabilities, and thus the system’s entropy, enables the prediction of the macroscopic properties of a physical system at equilibrium. This predictive capacity hinges on the knowledge of the a priori probabilities of observing the states of the system, given by the Boltzmann distribution. Unfortunately, the successes of equilibrium statistical mechanics are hard to replicate out of equilibrium, where the a priori probabilities of observing states are, in general, not known, precluding the naïve application of common tools. In the last decade, exciting developments have occurred that enable direct numerical estimation of the entropy and density of states of athermal and non-equilibrium systems, thanks to significant methodological advances in the computation of the volume of high-dimensional basins of attraction. Here, we provide a detailed account of these methods, underscoring the challenges present in such estimations, recent progress on the matter, and promising directions for future work.
Introduction: The Challenge of Non-Equilibrium Entropy
In classical statistical mechanics, predicting the behavior of a system is straightforward when the a priori probabilities—such as the Boltzmann distribution—are known. However, for non-equilibrium systems like granular materials (e.g., sand piles) or amorphous solids (e.g., glass), no such standard formula exists. Calculating entropy in these systems requires counting every possible stable arrangement, or microstate. For systems with even a small number of particles, this number is so vast that direct counting is numerically intractable.
Furthermore, these systems exist in high-dimensional configuration spaces where geometry is counter-intuitive. Most of the "volume" of a state resides in thin, spiky "tentacles," making simple random sampling fail due to the curse of dimensionality. To address this, researchers have developed a framework that shifts the problem from counting discrete states to sampling the high-dimensional volumes of their basins of attraction.
Methodology: Basin Sampling and Integration
A basin of attraction is the set of all initial conditions that lead a system to settle into a specific stable state or local minimum. By measuring the volume of representative basins, researchers can estimate the total entropy of the system.
Key techniques in this approach include:
- Turning Counting into Integration: Using the identity $V = \Omega \langle v \rangle$, where $V$ is the total configuration space and $\langle v \rangle$ is the average basin volume, the problem becomes one of high-dimensional integration.
- Biased Random Walks (Umbrella Sampling): To map a basin's volume, a "walker" is tethered to the basin's center with a virtual spring. Varying the spring's stiffness forces the walker to explore the narrow regions of the basin that uniform sampling would miss.
- Statistical Estimators: Tools like the Multi-State Bennett Acceptance Ratio (MBAR) and Parallel Tempering allow researchers to stitch together data from many different random walks into a single, accurate volume measurement.
Key Insights: The Geometry of Disorder
This methodology has revealed that in high dimensions, basins of attraction resemble "spiky urchins." Most of their volume is concentrated in long, thin spikes rather than a central core. This geometric insight explains why traditional "hitting" methods for counting states are ineffective.
Additionally, the framework has been used to test Edwards’ Hypothesis, which suggests that all jammed configurations of a granular material are equally likely. Numerical tests show this hypothesis is only strictly true at the exact point of unjamming. Above this threshold, a hierarchical landscape exists where low-energy states have significantly larger basins than high-energy states, following a power-law relationship between volume and pressure.
Results and Performance
The effectiveness of these methods was evaluated using high-dimensional hypercubes and Hard-WCA (Weeks-Chandler-Anderson) disks. Key results include:
- Dimensionality Scaling: The biased sampling method scales linearly with the number of dimensions ($d$), making it feasible for systems with thousands of particles. It successfully measured volumes in up to 1,000 dimensions, where naive methods fail after 30.
- Entropy Convergence: Researchers demonstrated that Gibbs-Shannon and Boltzmann entropy converge at the unjamming density ($\phi_J \approx 0.82$), supporting the equiprobability of states at that specific transition.
- Efficiency: The method can reconstruct the density of states to a precision requiring $10^{120}$ random samples using only $10^8$ points.
Future Directions
While computationally intensive due to the need for frequent steepest-descent calculations, this framework offers a robust path for exploring non-equilibrium systems. Potential applications include:
- Deep Learning: Measuring the "flatness" and volume of minima in neural network landscapes to understand generalization.
- Ecology: Applying the method to Lotka-Volterra models to determine the stability of complex ecological networks.
- Ensemble Extensions: Moving toward constant-pressure (isobaric) ensembles by utilizing enthalpy-like landscapes.
Summary updated