Abstract
The energy landscape picture is a central tool to study many-body systems. In particular, the energy landscapes of glass-forming liquids, jammed packings, constraint satisfaction problems, or neural networks contain a plethora of minima corresponding to competing states. Due to their complexity, these landscapes resist analytical treatment and must be studied numerically. We focus on jammed soft spheres, a paradigmatic model of glasses and granulars, to expose the limitations of standard numerical methods in resolving the true structure of energy landscapes. We show that CVODE is the ODE solver with the best time-for-error trade-off, outperforming commonly used steepest-descent solvers by several orders of magnitude. Using this numerical approach, we provide unequivocal evidence that optimizers widely used in computational studies destroy all semblance of the true landscape geometry, even in moderately low dimensions. Employing a range of geometric indicators, both low- and high-dimensional, we show that earlier claims on the fractality of basins of attraction of minima originated from the use of inadequate mapping strategies. In reality, the basins of attraction of soft sphere packings are smooth structures with well-defined length scales, a result that likely extends to a much broader family of problems.
Introduction: The Energy Landscape Problem
In fields ranging from materials science to machine learning, researchers study "energy landscapes"—high-dimensional maps of a system's potential energy. A key feature of these landscapes is the basin of attraction: the set of all starting configurations that settle into a specific local minimum when following the path of steepest descent. For years, computational studies suggested that these basins have fractal boundaries—infinitely complex, jagged edges where even a tiny nudge to an initial state could lead the system to a completely different minimum. This "fractality" was used to explain chaotic behavior and the difficulty of optimization.
However, calculating these basins requires solving a "stiff" ordinary differential equation (ODE). Because this is computationally expensive, most researchers have used heuristic optimizers like FIRE or L-BFGS that prioritize speed over mathematical trajectory accuracy. This research investigates whether the basins are truly fractal or if these numerical shortcuts have introduced artifacts.
Methods: Benchmarking the Path to Minima
The study compares several numerical strategies for mapping basins in a system of repulsive soft disks:
- Heuristic Optimizers: FIRE (Fast Inertial Relaxation Engine) and L-BFGS, which are standard in the field but do not strictly follow the steepest-descent path.
- Gradient Descent (GD): A simple approach that follows the gradient but is notoriously unstable and slow for "stiff" problems.
- CVODE: A sophisticated, high-performance ODE solver designed for stiff equations, used here as the "ground truth" with tight error tolerances ($10^{-14}$).
Key Insights: The Optimizer Mirage
The primary finding is that common optimizers like FIRE and L-BFGS "scramble" the landscape. Because they use momentum or curvature estimates to speed up, they often "jump" over ridges they should have stayed behind. This creates a "confetti-like" pattern of incorrectly tagged points that mimics fractality.
As shown in the benchmarking data, the accuracy of these heuristic optimizers drops exponentially as the number of particles $N$ increases. For systems larger than 64 particles, these optimizers almost never map a point to its true steepest-descent minimum. When using the high-precision CVODE solver, the box-counting dimension of basin boundaries is $\approx 1.1$ (nearly smooth), whereas heuristic optimizers report $\approx 1.3–1.4$ (distinctly fractal).
Quantitative Evidence for Smoothness
To further debunk the fractality claim, the researchers employed geometric "stress tests," such as measuring the survival probability $P_{in}(R)$—the likelihood a system returns to the same minimum after being displaced by a distance $R$.
In a truly fractal, scale-free system, one would expect power-law behavior. Instead, the data shows that $P_{in}(R)$ follows a stretched exponential decay. This implies the existence of a characteristic length scale, which is a hallmark of smooth, non-fractal structures. Furthermore, the volumes of the basins follow log-normal distributions rather than scale-free power laws.
Conclusion and Future Outlook
This work demonstrates that the "fractal" basins of attraction reported in previous literature are numerical artifacts caused by integration errors in fast optimizers. While CVODE is more accurate, it scales as $N^{5/2}$, making it significantly more expensive than the $N^{3/2}$ scaling of simpler optimizers.
While the study focused on soft sphere packings, the results suggest that the smooth nature of basins likely extends to other complex systems, including neural networks and constraint satisfaction problems. Future work must address the challenge of developing "good enough" optimizers that can preserve the true geometry of the landscape without the full computational cost of a stiff ODE solver.
Summary updated