Soft Modes in Oscillator Networks That Bind Features

Some neural networks group features into objects by letting units "synchronize." Each unit is an oscillator, a small vector that rotates, and units describing the same object pull their vectors into alignment. The settled, aligned state is a minimum of an energy function. We propose to examine these minima the way physicists examine glasses. The question is how the energy changes when the settled state is nudged in each possible direction, and which directions cost almost nothing.

In glasses and jammed packings, these nearly free directions come in excess. They cluster around "soft spots," and they mark where the material will give way first. Our claim is that trained binding networks have the same kind of soft spots. If so, those spots locate the places where a grouping is fragile and about to flip, such as a Sudoku cell that is about to go wrong or a boundary where two objects are about to merge. They should do so even when the network's current state looks confidently synchronized. The result would be a physically grounded diagnostic for binding failures and a cheap way to repair them at test time.

Core Claim

The model we study is Artificial Kuramoto Oscillatory Neurons (AKOrN). It replaces each neuron with a unit vector $\mathbf{x}_i$ on a sphere, couples units through learned weights $J_{ij}$, and lets the dynamics relax toward low values of $E = -\tfrac12\sum_{ij}J_{ij}\,\mathbf{x}_i\cdot\mathbf{x}_j$, possibly with an added input term. Low energy tracks correct groupings in that paper, including Sudoku solutions and segmented objects. What nobody has computed is the curvature of $E$ at the converged state, meaning the Hessian restricted to the product of spheres, or the distribution of its eigenvalues $\omega^2$. Physicists call that distribution the vibrational density of states $\mathcal{D}(\omega)$.

Glass physics says what to look for. A crystal follows Debye's law, with few low-frequency modes. Disordered solids carry a surplus of such modes, called the Boson peak. Near the rigidity threshold this surplus is set by constraint counting (Wyart 2005; Geometric origin of excess low-frequency vibrational modes). The surplus onset frequency scales as $\omega^*\sim\delta z$, where $\delta z$ is the excess of stiff contacts per unit over the isostatic count, the count at which the network is just barely rigid. The low-frequency modes are quasilocalized, with an unstable "buckling" core held in place by a stable elastic shell (Spatial structure of quasilocalized vibrations). Even systems with many total contacts can stay close to marginal. They do this by splitting into a near-isostatic backbone of stiff contacts plus weak contacts (Excess Vibrational Modes and the Boson Peak in Model Glasses; Lin et al.).

We make three hypotheses. First, trained AKOrN minima show a low-frequency excess over a matched reference. Second, the softest non-trivial modes are quasilocalized with a core-shell structure. Third, and most important, those cores sit on the units the model will get wrong, or would get wrong under a small perturbation. This is a statement about how the state responds to a push, not about how uncertain it currently looks. At a well-converged minimum every group is aligned, so phase dispersion is near zero everywhere, yet the minimum can still be marginal. That gap between looking settled and being stable is what the proposal targets.

Research Program

The first step is to build the spectral probe. For converged AKOrN states on Sudoku, CLEVRTex, and the ColorShape dataset from Formalizing the Binding Problem, we compute the lowest few hundred eigenpairs of the Riemannian Hessian. We use Lanczos iteration, which needs only Hessian-vector products from automatic differentiation, so the full matrix is never formed.

Two technical caveats must be handled before the probe is meaningful. First, AKOrN's couplings need not be symmetric, and its natural-frequency term $\mathbf{\Omega}_i$ makes the dynamics differ from pure descent on the energy. We therefore report two spectra: the Hessian of the symmetrized energy, and the spectrum of the Jacobian of the actual update map at its fixed point. We treat agreement between the two as a precondition, not an assumption. Second, global rotations of all oscillators cost zero energy by symmetry. These trivial zero modes are projected out.

From the spectra we measure four things. The first is $\mathcal{D}(\omega)$ against two references. One reference uses the same architecture with random, untrained couplings. The other uses the same trained couplings with the input-driven term amplified until the state is strongly pinned, which plays the role of a "crystal" reference. The second measurement is each mode's participation ratio, the effective number of units it moves. The third is the radial energy profile around each mode's peak unit, to test for a core-shell structure. The fourth is a stiff-contact count. We threshold couplings by their contribution to the Hessian, count stiff constraints per oscillator against its $N-1$ rotational degrees of freedom, and obtain $\delta z$.

The second step tests whether soft spots predict errors. For each input we assign every unit a softness score, its summed squared amplitude over the lowest $k$ non-trivial modes. We then ask whether softness predicts wrong Sudoku cells and mis-assigned segmentation pixels.

The third step uses soft modes to repair errors. We displace the state along the softest modes, re-relax it, and keep the lowest-energy result. This is a directed way to escape marginal minima, and it spends the same compute that AKOrN already spends on extra test-time iterations.

Baselines

Several baselines address different questions, and each is compared at a fixed budget of dynamics steps and forward passes, with hyperparameters tuned on the same validation split.

The naive reference is AKOrN with no intervention: the checkpoint, the number of steps, and the output are left exactly as released. On out-of-distribution Sudoku with fewer given clues, AKOrN reports 89.5% accuracy, against 30.3% for a recurrent Transformer. That 89.5% is the number our repair procedure must beat. For CLEVRTex, the file describes AKOrN only qualitatively, as competitive with slot-based models, so we found no reported AKOrN mIoU on Lacuna. We will reproduce AKOrN ourselves. As external anchors we cite the ClevrTex benchmark's reported IODINE mIoU of 29% and GNM mIoU of around 42% (ClevrTex).

The strongest competitor for repair is AKOrN's own test-time scaling: spend the extra budget on more relaxation steps. This is the baseline that already exists in the cited work, and it is strong because more iterations are known to help on hard out-of-distribution instances. A second repair competitor is random restarts, where the state is perturbed along random directions of matched norm, re-relaxed, and the lowest-energy result kept.

The random-restart baseline also serves as the key ablation. It holds everything fixed except the choice of perturbation direction, so it isolates whether the Hessian's structure matters. Beating it means higher out-of-distribution Sudoku accuracy, or higher CLEVRTex mIoU and foreground Adjusted Rand Index, at an equal step budget, with confidence intervals over at least five seeds.

For error prediction, the strongest competitors are cheap uncertainty signals computed from the same state. These are local phase dispersion, each unit's contribution to the energy, the per-unit speed of convergence, and the variance of the output under input noise. Softness counts as winning only if it raises the area under the ROC curve for predicting errors beyond a logistic combination of all four cheap signals. It must also do so specifically on errors made in states that look converged, where dispersion is low.

For the cosine-binding contrast on Pascal VOC, Binding Dynamics in Rotating Features reports a mean-best-overlap score of 0.460 for χ-binding and 0.449 for cosine binding. We will use these as a reference point for grouping quality, not as a target.

Why not just…?

Why not just use local phase dispersion as the uncertainty map? It is far cheaper, and for units that have not converged it will probably work as well as softness. This is the main threat to the error-prediction claim. The distinction we test is that dispersion reads the current state, while the Hessian reads how that state responds to a push. Our decisive analysis therefore restricts attention to units with dispersion below a threshold. If softness adds nothing on those units, the error-prediction claim fails and we will say so. In that case the spectral work reduces to a descriptive physics result.

Why not just run more iterations? The AKOrN authors show that extra steps help. However, running the same dynamics longer cannot leave a basin it has already settled in. Soft-mode kicks are designed exactly for such settled but wrong states. The matched-budget comparison decides this directly. We also report accuracy separately on instances where more iterations stop changing the answer.

Why not just use Slot Attention or self-attention instead? This proposal does not argue that oscillators are the best way to group features. It asks what the stability structure of an energy-based grouping model reveals. Slot models still matter as context: ClevrTex shows them splitting textured scenes into arbitrary tiles, and Berman & Deng document related failures of self-attention models on grouping tasks. As a check on the CLEVRTex camouflage split, where objects share materials with the background, we will test whether tiling errors coincide with extended, rather than localized, soft modes. If they do, tiling would correspond to losing rigidity across whole regions, which is the analogue of the length scale $l^*$ below which continuum elasticity breaks down.

Proposed Experiments

The decisive checks come in four parts.

The first check asks whether trained AKOrN minima show a statistically clear low-frequency excess over the pinned reference. It also asks whether the excess grows with task difficulty, measured as fewer Sudoku clues or more CLEVRTex objects. If there is no excess, we stop and report a negative result on the analogy itself.

The second check asks whether the onset frequency $\omega^*$ scales linearly with the measured $\delta z$. We vary $\delta z$ by pruning couplings and by using checkpoints taken at different points in training. The physics predicts a linear law; a clearly different exponent would itself be informative.

The third check is the error-prediction test on low-dispersion units, carried out on out-of-distribution Sudoku, CLEVRTex and its out-of-distribution and camouflage splits, and on ColorShape illusory conjunctions. Every result is compared against the combined cheap-signal baseline.

The fourth check is the matched-budget repair comparison against more iterations and against random restarts, with out-of-distribution Sudoku accuracy measured against the 89.5% reference as the headline number.

The Hessian-vector computations cost a few times a forward pass per Lanczos step. We will report wall-clock time and memory alongside accuracy, since Binding Dynamics in Rotating Features shows that grouping dynamics can raise memory use from 240 MB to 8,400 MB.