Surfing on minima of isostatic landscapes: avalanches and unjamming transition

Silvio Franz, Antonio Sclocchi, Pierfrancesco UrbaniJournal of Statistical Mechanics Theory and Experiment 2021 · DOI · arXiv

Abstract

Recently, it has been shown that optimization problems in both infinite and finite dimensions can display isostatic phases. In these phases, local minima of the cost function are marginally stable configurations characterized by non-linear excitations. This work introduces an athermal adiabatic algorithm to explore these rough high-dimensional landscapes continuously. Focusing on the spherical perceptron with a linear cost function (hinge loss), we investigate the statistics of avalanches triggered when local minima are destabilized. We find that plastic rearrangements are power-law distributed and characterize the critical properties of the unjamming transition, revealing logarithmic scaling in energy and pressure.

Introduction

In high-dimensional optimization, common in physics and machine learning, the landscape of the cost function often dictates the efficiency of search algorithms. A particularly intriguing regime is the "isostatic" phase, where the number of active constraints exactly matches the number of degrees of freedom. In this phase, local minima are not merely sub-optimal; they are marginally stable. A vanishingly small perturbation can trigger a cascade of rearrangements, or "avalanches," similar to those observed in granular materials and structural glasses. While theoretical frameworks have predicted these phases, continuous exploration of such landscapes has remained a challenge.

The Athermal Adiabatic Algorithm

To address this, we developed an "athermal adiabatic" algorithm. Unlike standard gradient descent, which may jump between disconnected minima, our approach follows a local minimum as a control parameter—pressure $p$—is varied. By treating optimization as a physical process of compression, we can drive the system through its landscape.

The algorithm utilizes a Lagrangian formulation where Lagrange multipliers represent the forces exerted by satisfied constraints. A configuration becomes unstable when a "contact force" leaves its stability interval. At this point, the algorithm simulates a "plastic event"—a sequence of rearrangements—until a new stable isostatic minimum is reached. To find the initial starting point at the jamming transition, we use a smoothed version of the Lagrangian to produce a valid configuration. This "surfing" technique allows for the precise study of transition statistics and critical behavior near the unjamming point.

Results and Discussion

The phase diagram of the spherical perceptron with hinge loss reveals a distinct "Critical Phase" where the landscape is non-convex and isostatic ($c=1$). In this region, the system exhibits marginal stability.

Phase diagram showing the Critical Phase where the landscape is non-convex and isostatic

Our simulations across various system sizes ($N=32$ to $2048$) show that the rearrangements triggered by instability are scale-free. The avalanche sizes follow a power-law distribution with a critical exponent $\tau \approx 1.41$, a value consistent with observations in sheared hard spheres. This suggests a deep universality between optimization landscapes and the physics of glasses.

Furthermore, we investigated the force distributions and pressure scaling at the jamming point. The distribution of forces $\rho(f/p)$ displays a characteristic pseudogap, and the pressure at first overlap $p_J(N)$ follows a logarithmic scaling law $\propto 1/\sqrt{\log N}$. These logarithmic corrections are a direct consequence of the linear interaction potential (hinge loss) and distinguish this system from those with harmonic potentials.

Force distributions at jamming and pressure scaling with system size N

Conclusion

By treating optimization as a physical process, we have uncovered the fractal nature of isostatic landscapes. The discovery of scale-free avalanches and specific logarithmic scaling laws near the unjamming transition provides a new lens through which to view high-dimensional optimization. Future work will explore the applicability of these findings to more complex architectures, such as deep neural networks, and investigate whether "annealing" through compression-decompression cycles can identify deeper, more stable minima.

Summary updated

Abstract

Abstract Recently, we showed that optimization problems, both in infinite as well as in finite dimensions, for continuous variables and soft excluded volume constraints, can display entire isostatic phases where local minima of the cost function are marginally stable configurations endowed with non-linear excitations [1, 2]. In this work we describe an athermal adiabatic algorithm to explore with continuity the corresponding rough high-dimensional landscape. We concentrate on a prototype problem of this kind, the spherical perceptron optimization problem with linear cost function (hinge loss). This algorithm allows to ‘surf’ between isostatic marginally stable configurations and to investigate some properties of such landscape. In particular we focus on the statistics of avalanches occurring when local minima are destabilized. We show that when perturbing such minima, the system undergoes plastic rearrangements whose size is power law distributed and we characterize the corresponding critical exponent. Finally we investigate the critical properties of the unjamming transition, showing that the linear interaction potential gives rise to logarithmic behavior in the scaling of energy and pressure as a function of the distance from the unjamming point. For some quantities, the logarithmic corrections can be gauged out. This is the case of the number of soft constraints that are violated as a function of the distance from jamming which follows a non-trivial power law behavior.
BibTeX
@article{franz2021surfing,
  author = {Silvio Franz and Antonio Sclocchi and Pierfrancesco Urbani},
  title = {{Surfing on minima of isostatic landscapes: avalanches and unjamming transition}},
  journal = {Journal of Statistical Mechanics Theory and Experiment},
  year = {2021},
  doi = {10.1088/1742-5468/abdc16},
}
Figure