Does a Surrogate's Compressibility Track How Itinerant Its Electrons Are?

Some magnetic materials sit between two textbook pictures. In one picture, magnetism comes from spins fixed to atoms that only interact with near neighbors. In the other, it comes from a sea of mobile electrons that carry magnetic influence across long distances. Neural networks are now used to speed up simulations of such magnets. One example is a convolutional network that predicts the torque that conduction electrons exert on each local spin. It replaces an electronic-structure calculation whose cost grows with the cube of system size at every time step (Convolutional neural networks for large-scale dynamical modeling of itinerant magnets). Separately, work on model compression shows that large networks carry a great deal of redundant capacity. They can lose most of their weights or activations with little loss, and the form of that redundancy (sparse versus low-rank) differs across parts of the network.

This note proposes to join these two lines in a controlled setting. We train torque surrogates on one family of lattice magnets and turn a single physical knob, the strength of the coupling between conduction electrons and local spins relative to electron hopping. This moves the target physics from short-range and local to long-range and mediated by mobile electrons. We then measure how the surrogate's redundancy changes along that axis. The hypothesis is that the knob changes not only how much a trained surrogate can be compressed but in which form. In the localized regime, surrogates should tolerate heavy sparse pruning but resist low-rank factorization. In the itinerant regime, the reverse should hold, because the long-range part of the electron-mediated interaction is concentrated on a few wavevectors. If that holds, a surrogate that splits its parameter budget between a sparse local stream and a low-rank global stream should beat a uniform convolutional network at equal size. It should do so precisely where the uniform network's fixed receptive field fails.

Core Claim

The physics premise is the standard itinerant-localized picture. When the coupling between conduction electrons and local spins, $J$, is strong compared with the hopping amplitude $t$, electrons are effectively slaved to the local spins and the induced interactions are short-ranged. When the coupling is weak, the interactions are mediated by the Fermi surface. They are long-ranged and oscillatory, dominated by a few momenta tied to the band filling.

Correlated-materials work treats this dual character as the central difficulty. Hybrid models for iron pnictides couple wide mobile bands to narrow localized ones through intra-atomic exchange (You et al.). Photoemission on intercalated dichalcogenides attributes their magnetism to this interplay (Zhen et al.). Static mean-field calculations fail partly because moment magnitudes fluctuate as electrons delocalize (Brännvall et al.). These papers motivate the axis we sweep, but none of them involves learned surrogates.

Our claim is about the surrogate. We define the sparse tolerance $s^*(J/t)$ as the largest fraction of surrogate weights that can be removed while keeping torque-prediction quality within a fixed tolerance. We define the rank tolerance $r^*(J/t)$ as the smallest rank of the factorized global operator that achieves the same quality. We predict that $s^*$ rises and $r^*$ grows as $J/t$ increases. We also predict that a dual-stream surrogate with budget allocation matched to these profiles dominates uniform architectures at fixed parameter count.

The architecture writes the torque on spin $i$ as

$$ \mathbf{T}_i = f_{\text{loc}}\big(\mathbf{S}_{\mathcal{N}(i)}\big) + \big[\,U\, g(V^{\top}\mathbf{S})\,\big]_i . $$

Here $\mathbf{S}$ is the full spin configuration and $\mathbf{S}_{\mathcal{N}(i)}$ is the patch of spins within a fixed neighborhood of $i$. The term $f_{\text{loc}}$ is a sparse convolutional stream. $V$ and $U$ are lattice-wide projection matrices of rank $r$, with translation-equivariant structure so that the stream transfers to larger lattices. The term $g$ is a small nonlinearity in the reduced space.

Research Program

The work has three parts. First, generate data. We use the same itinerant-magnet setting as the convolutional surrogate paper, with numerically exact torques from the Kernel Polynomial Method (KPM), a standard way to compute electronic responses without full diagonalization. We sweep roughly five values of $J/t$ and two band fillings. For each setting we record the physical torque-response range directly from KPM. This gives an independent measure of locality.

Second, run a redundancy census. Uniform ResNet surrogates are trained at four widths per setting. On each trained model we measure the following:

  • Critical sparsity, using top-$p$ activation thresholding in the style of Szatkowski et al..
  • Effective rank of intermediate activations.
  • Tolerance to magnitude pruning and to second-order one-shot pruning in the style of SparseGPT.
  • Tolerance to SVD factorization.
  • For the smallest models, where the full Fisher information matrix can be computed exactly, the off-diagonal coupling measure LOGDET from Perez et al..

That paper shows that pruning vision networks reduces weight coupling until density falls below about 10%, where coupling rises again. We ask whether the density at which coupling turns back up also shifts with $J/t$. If it does, that would locate the itinerant regime as one where the remaining weights must coordinate early.

Third, build and test the dual-stream surrogate. Its allocation between streams is set either from the census or by a learned budget penalty. It is compared against the baselines below at matched parameter counts.

The surrogate paper used its network's effective receptive field to find that electron-mediated torque falls off beyond the sixth-neighbor shell. It treated the network purely as a black box and assumed locality throughout. We test where that assumption breaks along $J/t$, and what the network's internal structure reveals there.

Baselines

The naive reference is the uniform convolutional surrogate of the itinerant-magnet CNN paper, trained at the target parameter budget from scratch. That paper reports that most predicted torque vectors fall within 10° of the KPM ground truth. It also reports that the network transfers from a 48×48 to a 96×96 lattice and stays stable over simulations five times longer than its training window. These are its reported values in its own model setting. No reported result was found on Lacuna for this surrogate across a sweep of coupling strengths, so we reproduce it at every setting.

The strongest compression competitor trains a uniform network several times wider and then prunes it to the target budget with second-order one-shot pruning. Where needed, it adds the sparse-dense-sparse reconstruction of Li et al., which is designed for cases where weight magnitudes do not separate important from unimportant weights. This baseline is strong because evidence across language (Boza et al.), vision and sequence models (Zhu et al.) and reinforcement learning (Obando-Ceron) says large-then-pruned beats small-dense at equal size. The only cross-domain anchor in the file is from SparseGPT: on OPT-175B, 2:4 structured sparsity moved perplexity from 8.34 to 8.74. That number shows how small the cost of pruning can be at extreme scale. It does not predict behavior for physics surrogates.

The strongest long-range competitor is a uniform convolutional network with a much larger receptive field, built with dilated convolutions or a spectral global layer. It is given the same parameter count and tuning budget as the other methods.

Two ablations isolate the key ingredient, which is budget allocation matched to the physics:

  • Single-stream variants. Each stream is trained alone, with the local stream alone and the global stream alone.
  • Mismatched allocation. The dual-stream model uses an allocation taken from the opposite end of the $J/t$ sweep.

All methods share data, total training steps, a fixed hyperparameter search of equal size, and the same torque-error objective.

Beating the baselines means two things. The dual-stream model must achieve lower torque angular error and longer stable rollouts than all baselines at the same parameter count in the itinerant regime. It must also be no worse than the pruned baseline in the localized regime.

Why not just…?

Why not just enlarge the receptive field of the existing CNN? This is the most plausible threat. A dilated or spectral layer may capture the long-range oscillatory tail with no explicit split. The matched-budget comparison settles it. If the large-receptive-field baseline equals the dual-stream model across the sweep, the architectural claim fails. The redundancy census would then still stand as a characterization result. Our expectation that it falls short rests on one argument. A uniform network must pay for long range with dense capacity at every location. The itinerant interaction, however, lives on a few wavevectors. That structure favors a low-rank operator over many dense weights.

Why not just train a small dense surrogate tailored to the budget? Prior work says large-then-pruned networks find better, less coupled solutions than small networks trained directly (Perez et al.; Zhu et al.). That evidence comes from classifiers and language models, not physics regression. The small-dense baseline therefore runs in every setting. If it matches the pruned baseline, this shows that the redundancy-scaling story does not transfer to physics surrogates. That would be a reportable negative result, not a failure of the study.

Why not just read locality off the KPM response directly and skip the network analysis? The physical response range is cheaper to obtain and is the ground truth for locality. We use it as a check, not as something to replace. The open question is whether network redundancy tracks it, which is a statement about learned models. We also include a physics-allocated dual-stream model, whose split is set from the measured response range rather than from the census. If that model matches the census-allocated one, the network diagnostic offers no extra design value. In that case we would say so and recommend the physics-allocated version.

Proposed Experiments

The first decisive check is the crossing. Plot sparse tolerance $s^*$ and rank tolerance $r^*$ against $J/t$ at each surrogate width. The hypothesis is supported only if the two curves move in opposite directions and the trend strengthens with width. A strengthening trend would mean that the over-parameterized slack is spent on structure matching the physics, not on generic redundancy. Flat or same-direction curves refute it.

The second check compares architectures at matched budgets of roughly $10^4$ to $10^6$ parameters. Metrics are:

  • Fraction of torques within 10°.
  • Relative torque error.
  • Transfer to 96×96 and 192×192 lattices.
  • Rollout fidelity of skyrmion density and spin structure-factor peaks under Landau–Lifshitz–Gilbert dynamics, the equation of motion for the spins.

The third check uses exact LOGDET on the smallest models. It tests whether coupling rises at higher remaining density in the itinerant regime. This would connect the Fisher-geometry picture to physical delocalization for the first time.

A positive outcome gives practical guidance on how to allocate surrogate capacity for correlated magnets. A negative outcome on the crossing shows that LLM-style redundancy findings do not explain physics surrogates. That result alone would correct an assumption the compression literature has so far left untested outside its home domains.