Does Pruning a Materials Model Break Its Physics First at Topological Phase Boundaries?

Stretching or squeezing a crystal can swap the order of its electronic energy levels. Physicists call this band inversion, and it can turn an ordinary insulator into a topological material, one whose surfaces conduct electricity in protected ways. Neural networks that predict a material's electronic Hamiltonian (the matrix whose eigenvalues are the allowed electron energies) now reach density-functional-theory accuracy in about a minute per structure. That speed makes it realistic to scan thousands of materials across many strain values. Running at that scale invites compression, and compression has a well-known failure: a performance cliff. Pruned networks hold their quality up to some sparsity and then collapse abruptly. Pruning studies also find that the first casualties are rare, long-tail cases. This note argues that strained materials near a topological transition are exactly such a long tail. Average Hamiltonian error will look fine after pruning while the model silently gets the band ordering wrong near the critical strain. That is the one regime where the prediction matters.

We propose to measure this effect and to fix it with a pruning procedure aware of the energy gap. In two sentences: a pruned model's Hamiltonian error only flips the predicted topology where the energy gap is smaller than the error, so the "topology cliff" should arrive at lower sparsity than the "error cliff" and should be predictable from the gap. Calibrating second-order pruning on structures sampled along strain paths and weighted toward small gaps should push that cliff back, at matched compression.

Core Claim

Average error metrics hide functional collapse in pruned networks. Understanding Performance Collapse in Layer-Pruned Large Language Models via Decision Representation Transitions showed this for language models: representation similarity stays high while task accuracy drops to zero. We claim the materials analogue is concrete and can be computed in advance.

Let $\Delta H(k)$ be the difference between the pruned and dense predicted Hamiltonians at crystal momentum $k$, and let $\Delta(k)$ be the true gap between the two bands that invert. By Weyl's eigenvalue perturbation inequality, every predicted eigenvalue moves by at most $\|\Delta H(k)\|_2$. The predicted band ordering is therefore guaranteed correct wherever

$$\|\Delta H(k)\|_2 < \tfrac{1}{2}\,|\Delta(k)|.$$

Near a strain-driven transition the gap passes roughly linearly through zero, $\Delta \approx (d\Delta/d\epsilon)(\epsilon-\epsilon_c)$, where $\epsilon$ is strain and $\epsilon_c$ the critical strain. The predicted critical strain should then shift by

$$\delta\epsilon_c \approx \frac{\|\Delta H\|_2}{|d\Delta/d\epsilon|}.$$

This gives a quantitative prediction for how the error in the predicted critical strain grows with sparsity. It also predicts which materials fail first: those whose gaps close slowly with strain.

The proposal is not to describe networks as crystals. That analogy is suggested in the file's gap list, and we reject it as untestable. Here strain physics enters as an evaluation regime and as the weighting for the calibration data. Pruning is the object of study.

Research Program

The model under study is an E(3)-equivariant Hamiltonian predictor. Such a network respects rotations, reflections and translations of the crystal. The specific model is the one in Advancing Universal Deep Learning for Electronic-Structure Hamiltonian Prediction of Materials (NextHAM), which learns a correction to a physics-based starting Hamiltonian, trains on a joint real- and momentum-space loss, and reports a gauge mean absolute error of 1.417 meV on its Materials-HAM-SOC dataset. If that model's weights are unavailable, we retrain an equivariant model of the same design on Materials-HAM-SOC, and its dense error becomes the reference.

We then build a strain-path test set by running our own density-functional calculations. Each path is a sequence of strained structures crossing a documented transition. Compressed black phosphorus inverts near $V\approx0.88V_0$, per Topological node-line semimetal in compressed black phosphorus. Biaxially stretched ReC₂H has two transitions, at 2.3% and 7.8%, per Switchable topological phase transition and nonlinear optical properties in a ReC2H monolayer. Further paths come from epitaxially strained stanene (Epitaxial growth of ultraflat stanene with topological band inversion), compressed CaP₃ (Topological nodal line semimetals in theCaP3family of materials) and uniaxially strained MnSb₂Te₄ (Topological phase transition in the layered magnetic compound MnSb2Te4: Spin-orbit coupling and interlayer coupling dependence). These papers supply ground truth; we reuse their transition points as targets.

The case set is deliberately heterogeneous. ReC₂H is useful because its first inversion happens at generic momenta, away from the symmetric points of the crystal. That is the case where errors are hardest to spot and ARPES cannot check easily. Black phosphorus has a tiny gap, under 10 meV, induced by spin-orbit coupling along its nodal line. That gap is comparable to plausible pruning errors.

We define three fidelity metrics:

  1. Error in the predicted critical strain $\epsilon_c$.
  2. The fraction of strain points where the band ordering, or the parity-based topological invariant where symmetry permits computing it, matches density-functional theory.
  3. Ordinary Hamiltonian error.

Ordinary error is reported separately for near-critical and far-from-critical structures.

The proposed method is a modified layerwise reconstruction in the style of SparseGPT: Massive Language Models Can be Accurately Pruned in One-Shot. That method minimizes $\|WX-\widehat W X\|^2$ over calibration inputs $X$, using a single shared inverse Hessian per layer. We change only $X$. Calibration inputs are drawn from strain paths of training-set materials, which are disjoint from the test materials. Each structure is reweighted by $1/\max(|\Delta|,\Delta_0)$, where $\Delta$ is its smallest gap at the inverting momenta and $\Delta_0$ is a floor. This implements the motivation drawn from A Simple and Effective Pruning Approach for Large Language Models and The Low-Resource Double Bind: An Empirical Study of Pruning for Low-Resource Machine Translation: calibration statistics decide what survives, and long-tail cases die first.

A secondary analysis tests whether the per-layer sensitivity profile of an equivariant network resembles the U-shaped outlier profile found in language models by Outlier Weighed Layerwise Sparsity (OWL): A Missing Secret Sauce for Pruning LLMs to High Sparsity. Nothing in the file says equivariant message-passing layers have such outliers, so we treat this as an open question rather than an assumption.

Baselines

The naive reference is the dense model. It sets the ceiling, and it also serves as a sanity check. Strained structures are out of distribution for a model trained mostly near equilibrium. If the dense model already misplaces $\epsilon_c$ by more than the strain step, the study becomes a study of the dense model, and we report that first.

The simplest pruning baseline is one-shot magnitude pruning. The Emergence of Essential Sparsity in Large Pre-trained Models: The Weights that Matter finds it matches iterative masks within a "safe" sparsity zone of roughly 30–60%. SparseGPT reports that it diverges on large language models beyond 30% sparsity.

We also run Wanda, which scores each weight by its magnitude times the norm of its input activation, without weight updates.

The strongest competitors are SparseGPT with standard calibration (random equilibrium structures from Materials-HAM-SOC) and OWL-style non-uniform layer allocation stacked on top of it. On LLaMA-7B at 70% sparsity, OWL reports WikiText perplexity 24.55 against 26.30 for SparseGPT and 85.77 for Wanda. These numbers come from language models only. There is no reported result found on Lacuna for pruning any Hamiltonian-prediction network, so the dense 1.417 meV is the only materials anchor.

All methods share the following controls:

  • the same number of calibration structures;
  • the same sparsity grid, from 30% to 90%;
  • per-layer sparsity matched, except where allocation is the variable under test;
  • the same hyperparameter search budget.

The isolating ablation is strain-path calibration with uniform weights versus gap-inverse weights. It separates "showing the model strained structures" from "emphasizing small gaps."

Beating the baselines means two things at matched sparsity. First, a statistically lower error in $\epsilon_c$ and higher ordering accuracy on held-out materials. Second, far-from-critical Hamiltonian error no worse than SparseGPT's. The method also counts as a success if the sparsity at which the topology cliff appears shifts upward by a full grid step.

Why not just…?

Why not just fine-tune the pruned model on near-critical structures? This is the most serious threat. Recovery training on a small budget often restores lost behavior. The language-model evidence from the decision-transition study, that fine-tuning cannot rebuild a path once it is cut too thin, may not transfer to Hamiltonian regression. We therefore run recovery fine-tuning on the same near-critical data, with compute matched to the calibration pass and with a second, larger budget. If fine-tuning closes the gap cheaply, the contribution narrows to diagnosis: the gap-based prediction of where and when pruned models fail. That remains useful for anyone screening with compressed surrogates.

Why not just add near-critical structures to the standard calibration set? That is precisely the uniform-weight ablation. If it matches the gap-weighted version, the weighting is unnecessary and we will say so. The perturbation bound predicts a difference only when near-critical structures are a small fraction of the calibration set. We test that by varying the fraction.

Why prune at all, when the dense model already cuts runtime by 97.4%? Fifty-eight seconds per structure, multiplied by dozens of strain points across thousands of candidates, is still a substantial cost. Dense equivariant transformers are also the expensive part of that pipeline. The deeper reason is scientific. Strain paths give a clean, physically labeled test of the claim, common in pruning work, that the cliff hits long-tail inputs first, and they quantify it. Language benchmarks cannot provide that test.

Why not just use the pruned model's ordinary error on a validation set as the safety check? Our central prediction is that this check fails. Near-critical error should rise far less than ordering accuracy falls, because a fixed error flips ordering only where the gap is small. Experiment 1 directly checks whether that divergence exists.

Proposed Experiments

Experiment 1 maps the sparsity curves for all baselines with two outputs on one plot: bulk Hamiltonian error and strain-path ordering accuracy. The decisive check is whether the ordering cliff occurs at lower sparsity than the error cliff. A second check is whether the measured $\delta\epsilon_c$ follows $\|\Delta H\|_2/|d\Delta/d\epsilon|$ across materials, for example by rank correlation over the five materials and multiple sparsities. If ordering and error collapse together, the core claim is false and we report a negative result.

Experiment 2 compares gap-weighted calibration against the baselines, against the uniform-weight ablation and against matched-compute fine-tuning. Evaluation uses held-out strain paths, with special attention to the generic-momentum inversion in ReC₂H. Experiment 3 measures layerwise sensitivity to test whether an outlier-style non-uniform allocation helps equivariant networks, or whether OWL's language-model finding fails to transfer.

The full study costs a few hundred additional density-functional calculations along strain paths plus pruning runs on a single GPU. That is modest relative to what it would show: whether compressed materials surrogates can be trusted where the physics changes.

References

  1. Understanding Performance Collapse in Layer-Pruned Large Language Models via Decision Representation Transitions
  2. Advancing Universal Deep Learning for Electronic-Structure Hamiltonian Prediction of Materials
  3. Topological node-line semimetal in compressed black phosphorus
  4. Switchable topological phase transition and nonlinear optical properties in a <mml:math xmlns:mml="http://www.w3.org/1998/Math/MathML"><mml:mrow><mml:msub><mml:mi>ReC</mml:mi><mml:mn>2</mml:mn></mml:msub><mml:mi mathvariant="normal">H</mml:mi></mml:mrow></mml:math> monolayer
  5. Epitaxial growth of ultraflat stanene with topological band inversion
  6. Topological nodal line semimetals in the<mml:math xmlns:mml="http://www.w3.org/1998/Math/MathML"><mml:msub><mml:mi mathvariant="normal">CaP</mml:mi><mml:mn>3</mml:mn></mml:msub></mml:math>family of materials
  7. Topological phase transition in the layered magnetic compound <mml:math xmlns:mml="http://www.w3.org/1998/Math/MathML"><mml:mrow><mml:msub><mml:mi>MnSb</mml:mi><mml:mn>2</mml:mn></mml:msub><mml:msub><mml:mi>Te</mml:mi><mml:mn>4</mml:mn></mml:msub></mml:mrow></mml:math>: Spin-orbit coupling and interlayer coupling dependence
  8. SparseGPT: Massive Language Models Can be Accurately Pruned in One-Shot
  9. A Simple and Effective Pruning Approach for Large Language Models
  10. The Low-Resource Double Bind: An Empirical Study of Pruning for Low-Resource Machine Translation
  11. Outlier Weighed Layerwise Sparsity (OWL): A Missing Secret Sauce for Pruning LLMs to High Sparsity
  12. The Emergence of Essential Sparsity in Large Pre-trained Models: The Weights that Matter