Many GPUs, especially consumer and workstation cards, run double-precision (64-bit) arithmetic at a small fraction of the speed of their low-precision matrix units, the Tensor Cores built for deep learning. Scientific codes that need 64-bit answers can close this gap in two ways. The first is emulation, which rebuilds 64-bit matrix products from several cheap 8-bit integer products. The second is iterative refinement: solve the system roughly in low precision, then repeatedly measure the error in high precision and correct it. Each approach has a weakness. Emulation pays for every extra "slice" of precision on every product. Refinement makes the user wait through a chain of serial correction passes, and that chain gets longer as the rough solve gets worse.
This proposal treats the number of emulation slices as a dial that is set while the solver runs. A linear solver factorizes the matrix with as few integer slices as it can get away with, watches how fast the refinement loop actually shrinks the error, and raises the slice count only when the observed shrinkage predicts that more refinement passes would cost more than a more accurate factorization. Prior work sets this precision once, ahead of time. The goal here is 64-bit-accurate linear solves on 64-bit-throttled GPUs, with no per-matrix tuning, faster than both fixed low-precision refinement and full 64-bit emulation.
Core Claim
Mixed-precision iterative refinement factorizes the matrix $A$ once at a working precision with unit roundoff $u_f$, the relative error of one rounding. It then repeats three steps: compute the residual $r_i = b - A x_i$ in 64-bit, solve for a correction using the cheap factors, and update $x_{i+1} = x_i + d_i$. Classical analysis gives a per-pass error shrinkage of roughly $\rho \approx c\,\kappa(A)\,u_f$, where $\kappa(A)$ is the condition number (how much the matrix amplifies input errors) and $c$ is a modest constant. Reaching 64-bit accuracy $u_{64}$ therefore takes about $k \approx \log u_{64} / \log \rho$ passes.
The Ozaki scheme, as implemented on integer Tensor Cores in DGEMM on Integer Matrix Multiplication Unit, lowers $u_f$ by a roughly fixed number of bits for each added slice $s$. The total cost is then
$$T(s) = F(s) + k(s)\,R,$$
where $F(s)$ is the time to factorize with $s$ slices, $R$ is the time of one serial refinement pass, and $k(s)$ is the number of passes. Because $\kappa(A)$ is unknown in advance, no fixed $s$ is right for every matrix. Our claim is that the shrinkage ratio measured over the first one or two passes, $\hat\rho = \|r_{i+1}\|/\|r_i\|$, estimates $\kappa u_f$ well enough to minimize $T(s)$ online. On 64-bit-throttled GPUs, this adaptive choice should beat every fixed precision profile across a wide range of condition numbers.
Research Program
The work builds directly on Energy-Efficient Iterative Refinement Using Dynamic Precision. That paper's solver, Trans-IR, does three things. It tracks cancellation bits at run time to decide when to switch precision. It inserts a nested inner refinement loop once the outer residual falls below an empirical threshold, $p=10$. It proves that the final accuracy check can be skipped. Trans-IR is 2.0–2.6× faster than fixed mixed-precision refinement on synthetic matrices with $n$ from 4K to 32K. However, it runs on a single CPU thread, uses XBLAS software arithmetic, and depends on a hand-set threshold. We keep its logic, which motivates this proposal and supplies the cancellation-tracking method. We replace both of its precision engines with GPU-native ones.
The factorization is a blocked LU, a factorization into lower- and upper-triangular factors. Its trailing-matrix updates, which account for nearly all the work, run through Ozaki integer-slice products with a run-time slice count $s$. The thin column panels are factorized in native FP32. The residual is a matrix-vector product, limited by memory bandwidth rather than arithmetic. We compute it in emulated 64-bit using two-component FP32 numbers, in the style of Representing Hyperbolic Space Accurately using Multi-Component Floats, where a value is stored as an unevaluated sum of a float and its rounding error. FP32 accumulation follows the round-to-nearest correction described in Recovering single precision accuracy from Tensor Cores while surpassing the FP32 theoretical peak performance.
The controller starts at a small $s_0$, chosen from the matrix's exponent spread, because the Ozaki paper identifies dynamic range as what drives the required slice count. After two passes it fits $\hat\rho$ and predicts the remaining time at the current $s$. It compares that with the time to refactorize at $s+1$, using an offline cost model of $F(s)$ measured once per GPU. It then picks the cheaper option. When restarting would be wasteful, a cheaper middle option is an inner correction loop at the current $s$, in the spirit of Trans-IR's restructuring step. If $\hat\rho \geq 1$, the controller escalates immediately.
We will also adapt Trans-IR's proof that the final check can be skipped to the emulated setting, since emulated 64-bit residuals carry their own rounding bound. A correctness guarantee matters more to adoption than a few percent of speed.
The target application is the stochastic reconfiguration solve in variational Monte Carlo, a sampling-based method for finding quantum ground states. Neural Quantum States in Mixed Precision moves sampling to 16-bit but reports that this ill-conditioned covariance solve must stay in native FP64. It therefore becomes the remaining serial bottleneck, and it is exactly the regime this method addresses.
Baselines
The naive reference point is native 64-bit LU from the vendor library (cuSOLVER on top of cuBLAS DGEMM) on the same GPU. It is what a practitioner gets without thinking. On a throttled card it sets the floor that any emulation or refinement scheme must beat. It needs no tuning.
The strongest competitor is fixed mixed-precision refinement: LU in FP32, optionally accelerated by Tensor Core FP32 emulation from Recovering single precision accuracy from Tensor Cores while surpassing the FP32 theoretical peak performance, followed by 64-bit residual passes until convergence. This is the "Mixed-IR" scheme Trans-IR improves on, it is simple, and for well-conditioned matrices it is hard to beat. We give it the same residual kernel and the same GPU, and we tune its stopping rule on the same calibration matrices. No reported result found on Lacuna for GPU mixed-precision iterative refinement time-to-solution. The nearest reported numbers are CPU ones from Trans-IR. At $n=32$K, its refinement phase took 705.6 seconds against 975.3 seconds for Mixed-IR, a 2.0–2.6× overall speedup. Beating this baseline means reaching the same backward error at least 1.5× faster on an RTX A6000 at moderate-to-high condition numbers, without losing on well-conditioned matrices.
The second strong competitor is full emulation without refinement: LU whose trailing updates use ozIMMU at the slice count needed for 64-bit parity. The file reports up to 6× over cuBLAS DGEMM for this on TITAN RTX and RTX A6000 (DGEMM on Integer Matrix Multiplication Unit). Beating it requires lower end-to-end time at equal accuracy. Any win must come from spending fewer slices than a fixed 64-bit level would.
Trans-IR ported to GPU keeps its $p=10$ threshold and cancellation-bit switching but uses our kernels. It isolates whether the slice dial adds anything beyond Trans-IR's restructuring logic.
The key ablation is an oracle fixed-$s$ solver: for each test matrix, an exhaustive sweep picks the best single slice count. Matching the oracle within about 10% shows the controller works. Beating a single best-on-average $s$ shows that adaptivity is what matters.
Why not just…?
Why not just run fixed FP32-factorization refinement on the GPU? For well-conditioned systems this is a real threat, and we expect it to tie our method there. The controller should then simply settle on the cheapest setting. The case for adaptivity is the ill-conditioned tail. Because shrinkage scales like $\kappa u_f$, the number of serial passes blows up as $\kappa$ approaches $1/u_{32}$, and the method fails beyond that point. Solinas et al. report that the stochastic reconfiguration matrices are ill-conditioned enough that lower-precision solves are unstable (Neural Quantum States in Mixed Precision). The condition-number sweep below settles where the crossover lies.
Why not just emulate full FP64 with ozIMMU everywhere? It is robust, but it pays the maximum slice count on every update. The slice count also grows with exponent spread, and the file names summation and bandwidth as the dominant cost once products are fast. If our oracle sweep shows the optimal $s$ is almost always the 64-bit level, refinement is pointless and we will say so.
Why not just reformulate the problem for better conditioning, as square-root bundle adjustment does? Square Root Bundle Adjustment for Large-Scale Reconstruction works on the Jacobian directly instead of forming $J^T J$, which squares the condition number. This lets FP32 match FP64 at twice the speed, but memory grows quadratically with observation density and runs out on final13682. Reformulation is problem-specific and orthogonal to our method. A dense covariance solve has no comparable structure to exploit, so we treat reformulation as complementary rather than competing.
Why not just tune one good fixed slice count offline? This is the oracle ablation. If a single global $s$ comes within 10% of the oracle on all matrix classes, the controller is unnecessary. The contribution then reduces to "emulated LU plus refinement," which is still a useful result.
Proposed Experiments and Decisive Checks
Hardware is an RTX A6000, the 64-bit-throttled target, and an A100 as a control. On the A100, native FP64 is strong, so we predict small or no gains and will report them. The synthetic suite follows Trans-IR's $n$ = 4K to 32K range. It adds controlled condition numbers from $10^2$ to $10^{14}$ and controlled exponent spread, so that the two known drivers of cost (conditioning for refinement, dynamic range for slicing) can be varied separately. The application suite consists of covariance matrices dumped from variational Monte Carlo training runs on the setups of Solinas et al.; there the measure is wall-clock per optimization step and ground-state energy agreement with a native-FP64 run.
The metrics are time and energy to a normwise backward error at the 64-bit level, number of serial refinement passes, and failure rate. Energy is measured with the board power counters, following the GFlops/W reporting of Ootomo and Yokota.
Three results decide the question. The adaptive solver must beat tuned fixed refinement by at least 1.5× on the A6000 for $\kappa$ between roughly $10^6$ and $10^{12}$. It must stay within 10% of the oracle-$s$ solver. It must have zero silent accuracy failures where the final check is skipped. If the first check fails, the slice dial is not worth its complexity. If the second fails, $\hat\rho$ is a poor proxy for $\kappa u_f$ and the controller design needs a different signal, such as Trans-IR's cancellation bits.
References
- DGEMM on integer matrix multiplication unit
- Energy-Efficient Iterative Refinement Using Dynamic Precision
- Representing Hyperbolic Space Accurately using Multi-Component Floats
- Recovering single precision accuracy from Tensor Cores while surpassing the FP32 theoretical peak performance
- Neural Quantum States in Mixed Precision
- Square Root Bundle Adjustment for Large-Scale Reconstruction