Neural-network quantum states are neural networks that output the amplitude and sign of a quantum wavefunction for each spin configuration. They now give the most accurate ground-state energies for two-dimensional frustrated magnets. A frustrated magnet is one where the lattice geometry, or competing couplings, prevents every pairwise interaction from being satisfied at once. The cost of this accuracy is large: the best result on the frustrated square-lattice $J_1$-$J_2$ model used a deep Vision Transformer trained on 20 A100 GPUs in full precision. Modern accelerators get most of their speed and memory savings from low-precision arithmetic (BF16, FP8). Nobody knows whether these wavefunctions tolerate it.
We argue that frustration itself sets how much precision is needed, and that we can predict where. In a frustrated system the energy of each sampled configuration is a sum of many terms with opposite signs that nearly cancel. The rounding error scales with the size of the terms, while the answer scales with what survives the cancellation. We propose to measure this cancellation, show that it predicts which parts of the training pipeline break under low precision, and build a mixed-precision pipeline that follows it. The pipeline keeps a few cancellation-sensitive operations in high precision and runs everything else on FP8/BF16 tensor cores.
Core Claim
Variational Monte Carlo trains the network by sampling spin configurations $\sigma$. For each sample it computes a "local energy":
$$E_{\rm loc}(\sigma)=\sum_{\sigma'} H_{\sigma\sigma'}\, e^{\log\psi(\sigma')-\log\psi(\sigma)}.$$
Here $H_{\sigma\sigma'}$ is the Hamiltonian matrix element connecting $\sigma$ to a neighboring configuration $\sigma'$, and $\log\psi$ is the network's complex log-amplitude. Suppose each log-amplitude carries an arithmetic error of size $\delta$. The relative error in $E_{\rm loc}$ is then roughly $R(\sigma)\,\delta$, where the cancellation ratio is
$$R(\sigma)=\frac{\sum_{\sigma'}|H_{\sigma\sigma'}\psi(\sigma')/\psi(\sigma)|}{|E_{\rm loc}(\sigma)|}.$$
This is a first-order error bound on a sum of terms that each carry relative error $\delta$.
The ratio behaves very differently with and without frustration. For an unfrustrated antiferromagnet the Marshall sign rule makes all off-diagonal contributions add with the same sign, so $R$ stays of order one. At $J_2/J_1\approx 0.5$ the signs are mixed, and $R$ should grow.
The claim has three parts. First, the precision needed to train is governed by the distribution of $R$, not by network size. Second, the damage is concentrated in a few places. These are the log-amplitude differences inside $E_{\rm loc}$, the complex output head that sets the sign, and the $2M\times 2M$ kernel solve in stochastic reconfiguration, a natural-gradient optimizer. Third, the dominant compute can run at FP8/BF16 with no loss of energy accuracy. That compute is the real-valued Transformer backbone and the Jacobian products.
This differs from prior "noise is fine" results on Ising hardware. The Ising-substrate RBM work shows that analog training of Boltzmann machines tolerates 10–30% component noise. BRIM finds that 12-bit couplings barely affect Max-Cut quality. Both concern classical, sign-free energy functions, where noise is absorbed by sampling. Our mechanism is the opposite: sign cancellation amplifies noise. So those results motivate the question but cannot answer it. The research file's Gap 5 flags this tension, and we turn it into a quantitative prediction.
Research Program
The method starts from two papers. The push-through identity paper rewrote stochastic reconfiguration so that a $P\times P$ metric with $P\approx 2.7\times10^5$, which would need over 1,000 GB, becomes a $2M\times 2M$ kernel with $M=6000$, about 0.57 GB. That work kept full precision on general GPUs, and its bottleneck moved to building and communicating the $M\times P$ Jacobian. The Shastry-Sutherland Transformer splits the network into a deep real-valued backbone and a single shallow complex output layer. This split is a natural precision boundary, because the backbone is ordinary real matrix multiplication and only the head encodes phase.
We will build the pipeline in NetKet 3. NetKet provides JAX compilation, Jacobian-vector products and chunked evaluation. We add per-operation precision control in six places: backbone matrix multiplications, attention, the complex head, the $\log\psi(\sigma')-\log\psi(\sigma)$ subtraction, the $E_{\rm loc}$ accumulation, and the Jacobian and kernel formation and solve.
Four analyses sit on top of the pipeline. We measure $R(\sigma)$ along training trajectories as a function of $J_2/J_1$. We run a sensitivity sweep that drops one component at a time to low precision. We test whether $R$ predicts which components fail and at what $J_2/J_1$. We also check whether the diagonal shift $\lambda$ in the kernel solve absorbs Gram-matrix rounding when $\lambda$ exceeds the rounding error of $\mathcal{O}\mathcal{O}^\dagger$; if so, the kernel can be formed in BF16.
The output is a precision map validated on three benchmarks: the 1D $J_1$-$J_2$ chain, the 2D $J_1$-$J_2$ square lattice, and the Shastry-Sutherland model. We will report throughput and memory on H100-class hardware.
Baselines
The naive reference point is uniform full-precision training (FP32, with FP64 local energies) using push-through stochastic reconfiguration. This is what Rende et al. ran. Their reported result is a ground-state energy per site of −0.497634(1) on the 10×10 $J_1$-$J_2$ lattice at $J_2/J_1=0.5$, with a Deep ViT of 267,720 parameters on 20 A100s.
We cannot afford that full run for every variant. We will therefore reproduce a reduced configuration: 6×6 and 8×8, with a smaller ViT and a fixed number of optimization steps. We will run one 10×10 full-budget comparison for the final pipeline only. Every variant shares the architecture, the sampler, $M$, $\lambda$, the learning-rate schedule and the seeds. Beating full precision means matching its energy within statistical error while using fewer GPU-hours or less memory to reach it. The 6×6 energy must also match exact diagonalization within the full-precision run's own error. For Shastry-Sutherland the reference is the reported relative error of about $10^{-5}$ on 6×6 (Transformer SSM paper).
The strongest competitor is uniform low precision via standard automatic mixed precision. This is BF16 everywhere with FP32 master weights, the recipe used for large language model training. It is the default any practitioner would try first, and it may already be good enough.
We run two isolating ablations. The first applies our precision map but lets the $E_{\rm loc}$ subtraction and accumulation drop to low precision. It tests whether protecting the cancellation is the key ingredient. The second keeps the local energy protected but lowers the complex head. It separates sign errors from amplitude errors.
Each ablation is repeated at $J_2/J_1=0$ as a frustration control. There the theory predicts that all variants should behave the same. For the 2D benchmarks we found no reported low-precision result on Lacuna, and none for NetKet GCNN ground-state energies.
Why not just…?
Why not just turn on BF16 automatic mixed precision and see? This is the real threat, and the experiment is designed to settle it. If uniform BF16 matches full precision at $J_2/J_1=0.5$ on 6×6 against exact diagonalization, the practical contribution shrinks to the diagnostic. The measured $R$ would then show why BF16 suffices, and the diagnostic predicts where FP8 fails. We expect uniform BF16 to be harmless at $J_2=0$ but to bias energies or raise the variance of $E_{\rm loc}$ near $0.5$. BF16 has about three significant digits. That precision is a problem once $R$ reaches $10^2$ or more, which is exactly the regime our diagnostic tracks.
Why not just stay in full precision and buy more GPUs? It works, as Rende et al. show, and NetKet 3 scales nearly ideally across MPI ranks. But the Shastry-Sutherland work reports that 18×18 runs take days to weeks. Their bottleneck is the Jacobian, and halving or quartering its bytes directly cuts memory and communication. The question is whether that saving is free, and our wall-clock-to-target-energy metric answers it.
Why not just drop stochastic reconfiguration for Adam, removing the expensive kernel entirely? The push-through paper reports that first-order optimizers converge poorly on frustrated landscapes. We include Adam at matched step cost as a sanity baseline. We expect it to trail the full-precision baseline in energy, so the mixed-precision kernel remains worth protecting.
Proposed Experiments
The first experiment is diagnostic. We track the distribution of $R(\sigma)$ during full-precision training on the 1D chain ($L=100$, where the 1D Transformer paper reports relative errors ≲0.1% for $J_2/J_1>0.5$) and on 6×6 and 8×8 square lattices. We sweep $J_2/J_1\in\{0,0.3,0.4,0.5,0.6\}$. The prediction is that the median and tail of $R$ rise sharply toward 0.5.
The second experiment is the per-component precision sweep described in the Research Program, at the same couplings. For each variant we report energy error against exact diagonalization, $E_{\rm loc}$ variance, and training stability.
The decisive check is correlational and falsifiable. The coupling at which each low-precision variant first departs from full precision should track the measured $R$ tail. It should not track network size, which we vary independently. If failure is unrelated to $R$, for example if uniform FP8 fails equally at $J_2=0$, the core claim is wrong.
The final experiment runs the resulting precision map on the 10×10 $J_1$-$J_2$ lattice and on 6×6 and 10×10 Shastry-Sutherland lattices. We report energy relative to −0.497634(1), memory for the Jacobian and kernel, and GPU-hours to reach the full-precision energy.
References
- Supporting Energy-based Learning with an Ising Machine substrate: a Case Study on RBM
- BRIM: Bistable Resistively-Coupled Ising Machine
- Transformer Variational Wave Functions for Frustrated Quantum Spin Systems
- A simple linear algebra identity to optimize large-scale neural network quantum states
- Transformer wave function for two dimensional frustrated magnets: Emergence of a spin-liquid phase in the Shastry-Sutherland model
- NetKet 3: Machine Learning Toolbox for Many-Body Quantum Systems