SVD-Free Onsager Correction for Learned Image Reconstruction Networks

Many imaging systems recover a picture from fewer, noisier measurements than pixels: an MRI scanner, a coded-aperture camera, a blurred and downsampled sensor. A popular fix is deep unfolding, which takes a fixed number of steps of a classical iterative solver and turns each step into a network layer with learnable parts. These networks are fast and accurate, but their internal errors become tangled with the measurement process from layer to layer. As a result, they tend to break when the measurement operator changes. Approximate message passing (AMP), an iterative estimator from statistical physics, avoids this with the Onsager correction, a memory term subtracted at each step. The term makes the error entering each denoising stage look like plain white Gaussian noise of known size. That is why learned AMP networks need fewer layers, can be trained one layer at a time, and allow a one-line scalar recursion ("state evolution") to predict their accuracy before they run.

The catch is that the cheap form of the correction only works for idealized random measurement matrices. The robust form, vector AMP, needs a singular value decomposition of the operator, which is cubic in cost and unavailable for most real operators that are only accessible through matrix-vector products. We propose to build an unfolded reconstruction network whose Onsager correction is computed from a handful of spectral moments of the operator. Those moments are estimated on the fly with random probes, so the network uses only forward and adjoint applications of the operator. Each layer's denoiser is a CNN trained for a known Gaussian noise level. The aim is the operator robustness of vector AMP at the cost of plain AMP, inside a learned network, on operators where no fast decomposition exists.

Core Claim

Recent work on rotationally invariant models shows that the Onsager coefficients of a long-memory message passing algorithm can be written as functions of the spectral moments $m_k = \frac{1}{N}\mathrm{tr}\big((\mathbf{A}^\top\mathbf{A})^k\big)$ through free cumulants. Approximate Message Passing for Multi-Layer Estimation in Rotationally Invariant Models uses this to replace per-layer SVDs in a multi-layer estimator, reporting 30× to 200× speedups over ML-VAMP at matched MSE. It does so only for synthetic Haar-distributed matrices, and without any learned components. Our claim is that the same moment-based correction, placed inside an unfolded network with learned CNN denoisers, gives three things together. It keeps each denoiser's input close to signal plus white Gaussian noise. It stays stable on ill-conditioned and structured operators where learned D-AMP collapses. It transfers across operators without retraining, because the operator enters only through its moments and its matrix-vector products. In schematic form, layer $t$ forms its denoiser input as

$$\mathbf{r}_t = \mathbf{A}^\top \mathbf{z}_t - \sum_{s<t} \beta_{t,s}\,\hat{\mathbf{x}}_s, \qquad \beta_{t,s} = \beta_{t,s}\big(m_1,\dots,m_{2t},\ \overline{\mathrm{div}}_1,\dots,\overline{\mathrm{div}}_{t-1}\big).$$

Here $\mathbf{z}_t$ is the current measurement residual, $\hat{\mathbf{x}}_s$ are past denoiser outputs, $\overline{\mathrm{div}}_s$ are the denoisers' average divergences (estimated by Monte Carlo as in learned D-AMP), and $\beta_{t,s}$ are the memory coefficients given by the free-cumulant recursion. The moments are estimated once per operator by Hutchinson trace estimation, which averages $\mathbf{b}^\top(\mathbf{A}^\top\mathbf{A})^k\mathbf{b}$ over random probes $\mathbf{b}$. That costs a few probes times $2T$ matrix-vector products for a $T$-layer network.

Relation to Prior Work

Three papers already join Onsager correction with unfolding, and each leaves the same gap open.

  • LAMP and LVAMP. Onsager-Corrected Deep Networks for Sparse Linear Inverse Problems unrolls AMP (LAMP) and vector AMP (LVAMP) with scalar shrinkage. It shows Gaussian layer errors on Q-Q plots and reaches −34 dB NMSE in 7 layers, versus 15 for LISTA. Its robust variant needs an SVD, and it has no image priors.
  • Learned D-AMP. Learned D-AMP puts a DnCNN denoiser inside AMP. It shows that layer-by-layer training is optimal under the Onsager decoupling, and it transfers from 20% to 5% sampling. It inherits AMP's fragility to non-i.i.d. operators.
  • D-VAMP. Denoising based Vector Approximate Message Passing makes plug-in denoising robust to condition numbers up to $10^4$, using BM3D and an SVD.

These papers motivate the proposal: they establish that Onsager decoupling buys depth efficiency, predictability, and transfer. The moment-based approach of the free-cumulant paper supplies the method. What none of them does is the combination we propose: learned image priors, robustness to ill-conditioning, and access to the operator only through matrix-vector products.

Mainstream unfolded networks such as ISTA-Net and Deep ADMM-Net have no correction term and train end-to-end. They are our main empirical competitors, not our method.

One theoretical limit constrains the work. The free-probability guarantee assumes Haar singular vectors, which a blur kernel or subsampled transform does not have. Whether the correction still whitens the error on such operators is an empirical question. We treat it as the decisive test rather than assume it.

Research Program

We will build a single-layer, image-domain version of the moment-based long-memory AMP. Each step contains one forward and one adjoint operator application, a CNN denoiser conditioned on the noise level that state evolution predicts, a Monte Carlo divergence estimate, and the memory correction above.

Numerical stability. High-order moments estimated from few probes are noisy, and long-memory recursions can amplify that noise. We will study probe count against depth, and we will try damping and truncating the memory to the last few iterates. The ill-conditioned synthetic ensembles can tell us whether truncation preserves robustness.

Training. Following learned D-AMP, the default trains each denoiser on Gaussian denoising at its predicted noise level. We will compare this with end-to-end fine-tuning in two settings: the $\beta_{t,s}$ held at their analytic values, or freed as learnable parameters. This directly answers the open question of whether end-to-end training repairs finite-size mismatches or destroys the decoupling that makes state evolution valid.

Instrumentation. Every layer is measured in two ways. Gaussianity of $\mathbf{r}_t - \mathbf{x}$ is checked with Q-Q statistics and the whiteness of its spatial autocorrelation. We also record the gap between the per-layer PSNR that state evolution predicts and the PSNR actually reached.

Baselines

Naive reference. The do-nothing reference is the adjoint-plus-denoise reconstruction: one application of $\mathbf{A}^\top$ followed by the same trained CNN. It shows how much any iteration adds.

Learned D-AMP. Learned D-AMP with standard scalar Onsager correction (Learned D-AMP) is the strongest existing cheap Onsager-corrected network. It reports 28.1 dB PSNR on Boat under coded diffraction measurements, against 27.2 dB for BM3D-AMP. We expect to match it on well-conditioned operators and must beat it clearly once the condition number exceeds about 10. That is where D-AMP is reported to collapse to roughly 7–8 dB (D-VAMP).

Unfolded D-VAMP with an exact SVD. This is LVAMP's linear stage with our CNN denoisers. It is the accuracy ceiling for robust Onsager correction, reported in its BM3D form at 20–24 dB PSNR for condition number $10^4$ (D-VAMP). It can only be run at sizes where the SVD is feasible, so we compare there. "Beating" it means coming within 0.5 dB of it while using only matrix-vector products, and then running at sizes it cannot reach.

ISTA-Net+. ISTA-Net reports 32.57 dB on Set11 at a 25% sampling ratio. It is the strongest uncorrected unfolded network. We run it twice: once trained on the test operator, and once trained with operators randomly drawn from the evaluation families. The randomized version is its fairest robustness setting.

Fairness and ablation. All methods share the BSD-500 training patches, the denoiser architecture and parameter count, the number of layers, a matched hyperparameter search budget, and a matched total count of operator applications. The key ablation removes the memory sum. A second ablation replaces it with the scalar AMP correction. Together these isolate the moment-based correction as the source of any gain.

Why not just…?

Why not just approximate vector AMP's linear step with a few conjugate-gradient iterations instead of an SVD? This is the most serious alternative. It also uses only matrix-vector products, so we include an unfolded conjugate-gradient D-VAMP as a baseline at a matched budget of operator applications. Our reason for expecting an advantage is that an inexact inner solve breaks the exact divergence that the vector Onsager term relies on. Our method spends its budget on spectral moments that the correction uses directly. If the conjugate-gradient version matches us in accuracy, stability, and state-evolution agreement at equal cost, then the honest conclusion is that moment-based correction is unnecessary. The experiment is designed so that this outcome would be visible.

Why not just train an uncorrected unfolded network on randomly drawn operators? Operator-randomized ISTA-Net+ may learn robustness without any theory. Learned D-AMP reports that unprincipled networks collapse when the sampling ratio shifts, but that evidence is for networks trained on a single operator. Our held-out operator families, especially blur kernels and condition numbers not seen in training, test whether randomization generalizes or merely interpolates. Predictable per-layer accuracy is something it cannot provide in any case.

Why not just use learned D-AMP with damping? Damping slows divergence on mildly ill-conditioned matrices, but it does not change the fact that the scalar correction is derived for i.i.d. matrices. We include damped learned D-AMP in the condition-number sweep. If damping alone holds up to $\kappa = 10^4$, our contribution shrinks to the transfer and prediction results.

Proposed Experiments

Synthetic study. The first experiment uses Bernoulli-Gaussian signals and ill-conditioned matrices with condition numbers swept from 1 to $10^4$. Some matrices have Haar right singular vectors, where theory applies. Others have deterministic DCT right singular vectors, where it does not. We measure NMSE against layer count, using LAMP's −34 dB in 7 layers (Onsager-Corrected Deep Networks) as the well-conditioned reference.

Image study. Training uses BSD-500 patches, with testing on Set11 and BSD68. The operators are i.i.d. Gaussian at 25% sampling, coded diffraction, Gaussian matrices with prescribed ill-conditioned spectra, and blur followed by 2× or 4× downsampling. Results are reported in PSNR, along with runtime and operator-application counts. Transfer is the key test: models trained on one operator family are evaluated, without retraining, on the others and at sampling ratios from 5% to 50%.

Decisive check. The decisive result is not peak PSNR. It is whether the gap between predicted and achieved per-layer PSNR, and the Gaussianity of the effective noise, stay small on structured non-Haar operators such as blur-downsampling at high condition numbers. If they do, cheap Onsager correction extends to realistic imaging operators. If they degrade while PSNR stays good, the method is a useful heuristic but not a principled one, and we will report it that way.