Many imaging problems, such as accelerated MRI, sparse-view CT and phase retrieval, are ill-posed: tiny measurement noise turns into large image errors unless the solver is regularized. Modern solvers regularize with a learned image denoiser, and their quality depends on a handful of knobs. These are the denoising strength, the step size and when to stop, and they are best changed from iteration to iteration. The two current ways of setting these knobs are expensive. One is a sweep over many full reconstructions. The other is training a policy network on ground-truth images, which fails when the scanner, sampling pattern or noise level differs from training. We propose tuning the knobs during a single reconstruction, using only the data the scanner produced. We hide a random subset of the measurements from the solver. At regular intervals we take a gradient step on the knobs so that the current reconstruction better predicts the hidden measurements.
This is the cheap online hyperparameter tuning used in deep learning, where the network is fit on one data split and its regularizers are adjusted by gradient steps on a second split. The difference is that the "validation set" here is part of the measurement vector itself. The question that decides whether the idea is worth anything is empirical. Error on held-out measurements is an imperfect stand-in for image error, because it is blind to image content that the scanner never measures. We do not know whether it is a good enough stand-in to choose iteration-dependent schedules that beat the best fixed setting, at a cost of a few reconstructions rather than dozens. The proposal tests this directly and is built so that a negative answer is still informative.
Core Claim
Consider a solver with plug-and-play (PnP) structure: an iterative method that alternates a data-consistency step with an off-the-shelf neural denoiser applied in place of a hand-written regularizer. Its per-iteration denoising strengths and step sizes, and its stopping point, can be set per instance by first-order hypergradients of a held-out-measurement loss. Done this way, the reconstruction should approach the quality of a schedule tuned with ground truth. We expect it to beat the best static setting found by held-out-measurement cross-validation at equal compute. It should also degrade more gracefully than a supervised tuning policy when the forward operator shifts.
Concretely, split the acquired measurement indices $\Omega$ into a fitting set $\Theta$ and a held-out set $\Lambda$. Let $x_t(\eta)$ be the PnP iterate after $t$ steps run on $y_\Theta$ alone, where $\eta = (\sigma_1,\mu_1,\dots,\sigma_T,\mu_T)$ is the schedule of denoiser strengths $\sigma_t$ and penalty/step parameters $\mu_t$. The knobs are updated to reduce
$$ \mathcal{L}_{\text{ho}}(\eta) = \big\| P_\Lambda\big(A\, x_t(\eta)\big) - y_\Lambda \big\|^2 , $$
where $A$ is the forward operator and $P_\Lambda$ keeps only the held-out measurement rows. When the noise is independent across measurements, the noise in $y_\Lambda$ adds a constant to this loss and leaves its gradient with respect to $\eta$ unbiased. The gradient is still blind to the null space of the full operator $A$, so how informative it is depends on the operator. That dependence is what we will measure.
Relation to Prior Work
Three intersecting lines of work motivate this proposal, and each leaves the same gap: a tuning signal that needs no ground truth.
TFPnP and its conference version Tuning-free Plug-and-Play Proximal Algorithm train a reinforcement-learning policy to pick $\sigma_t$, $\mu_t$ and the stopping step for PnP-ADMM. They show that dynamic schedules beat the best fixed grid-searched parameters. However, their reward is a gain in PSNR against clean images, and the policy is trained offline on a fixed operator family. We keep their evidence that schedules matter and their use of a differentiable denoiser. We replace the reward with a measurement-only loss and the policy with per-instance gradient steps.
On Learning the Optimal Regularization Parameter in Inverse Problems learns a single scalar regularization weight by empirical risk minimization over a grid. It uses paired ground truth and gives sharp rates. Deep Equilibrium Models versus Bilevel Learning treats regularizer learning as a two-level optimization with a supervised upper-level loss. It also finds that the variational structure makes training less sensitive to step size and regularization weight. Both constrain our design. The first suggests that one well-chosen scalar may already capture most of the gain, which is why static cross-validation is our key ablation. The second suggests keeping the PnP update close to a gradient-like map so that hypergradients stay well behaved.
The method itself is borrowed. Scalable Gradient-Based Tuning of Continuous Regularization Hyperparameters updates regularization hyperparameters every ten weight steps using greedy one-step validation gradients, at roughly 30% overhead, and matches grid search. Self-Tuning Networks shows that learned dynamic schedules beat fixed ones. Both assume a separate validation dataset. In single-instance inversion there is none, and measurement splitting provides it.
Measurement splitting comes from Self-Supervised Deep Equilibrium Models, which partitions k-space so that network weights can be trained without ground truth. Its solver hyperparameters are still tuned by hand. We use the split for hyperparameters rather than weights.
Two papers on untrained network priors bound what a fixed stopping rule can achieve. Early Stopping for Deep Image Prior stops on a variance signal within 0.2–1.5 dB of the oracle peak, but adds its own window and patience parameters and tunes nothing continuous. Deep Image Prior Subspaces removes the need for stopping by restricting capacity, at the cost of pretraining and some over-smoothing. Finally, COEP shows that annealing the regularization weight avoids bad local minima, but leaves the target weight to be chosen by hand. Our held-out loss could choose it.
Research Program
The method has two phases.
In the first phase, PnP-ADMM with a differentiable U-Net denoiser runs on $y_\Theta$, with $\Lambda$ a random 10–20% of the measurements. The sampling of $\Lambda$ is matched to the acquisition pattern; for MRI, held-out lines are drawn with the same density as the sampling mask. Every $k$ iterations we compute $\mathcal{L}_{\text{ho}}$ and backpropagate through only the last few iterations, which is the greedy short-horizon approximation of Luketina et al. The resulting gradient updates the current schedule entries $(\sigma_t, \mu_t)$, which are parameterized as a smooth low-dimensional curve in $t$ rather than free per-iteration values. Stopping is set by the minimum of the held-out loss over iterations. This replaces both the discrepancy principle's need to know the noise level and the window and patience parameters of variance-based stopping.
In the second phase, the learned schedule is replayed on the full measurement vector $y_\Omega$. Fitting on $\Theta$ sees fewer measurements than the final solve, so the schedule tuned on $\Theta$ is slightly mis-calibrated. We handle this with a single correction factor on $\sigma$ that depends on the measurement fraction. We test this correction against two alternatives: keeping the reconstruction from $\Theta$ alone, and averaging the schedules learned from two disjoint splits.
The total cost target is at most three times one plain reconstruction. For comparison, Luketina et al. report that random search needs 10–50 full runs.
The analysis has three parts. The first is a precondition check. For each operator we sweep fixed settings and schedules, and compute the rank correlation between held-out-measurement loss and true PSNR. We expect high correlation for random-like operators such as variable-density MRI masks and phase retrieval. We expect it to weaken for structured operators with large null spaces, such as limited-angle CT. Characterizing this dependence is a contribution in its own right.
The second part locates the gain: whether it comes from the schedule, the stopping point, or the scalar level. We isolate each by freezing the others.
The third part extends the same held-out-measurement hypergradient to untrained network priors, tuning per-layer weight decay and a total-variation weight within one fit. This tests whether the signal generalizes beyond PnP.
Baselines
The do-nothing reference is PnP-ADMM with a fixed schedule and iteration count, tuned once on a calibration phantom and then frozen. This is what practitioners actually ship, and any method costing more than one reconstruction must clearly beat it. It runs with the same denoiser and the same iteration cap.
The strongest existing competitor is TFPnP's supervised reinforcement-learning policy. On MICCAI CS-MRI at 4x acceleration, Tuning-free Plug-and-Play Proximal Algorithm reports 36.11 dB PSNR, against 35.80 dB for the tuned IRCNN baseline. At 2x it stops in 5 iterations, where other methods need 30–70. We will retrain TFPnP with its published setup and the identical denoiser, so the comparison isolates the tuning mechanism. On the in-distribution operator, matching it within about 0.3 dB without ground truth counts as success. Beating it requires exceeding it on shifted operators: a different acceleration factor or mask density than it was trained on.
The key ablation is static held-out cross-validation. It uses the same split and the same held-out loss, but runs a grid search over a single scalar pair $(\sigma, \mu)$ and the stopping step. The number of reconstructions is chosen to match our method's wall-clock cost. This isolates whether gradient-tuned schedules add anything beyond simply having a ground-truth-free validation signal. The proposal's central claim stands only if we beat it by a margin larger than the run-to-run variation over splits.
The upper bound is an oracle grid search, static and scheduled, that selects using ground-truth PSNR. Two classical references complete the set. One is the Morozov discrepancy principle, run both with the true noise level and with a mis-specified one, as treated in Convergence Analysis of Ensemble Kalman Inversion. The other is variance-based stopping from Early Stopping for Deep Image Prior, applied to the PnP iterates.
Why not just…?
Why not just do cross-validation over a few fixed settings with the held-out measurements? This is the real threat. If one scalar captures most of the benefit, as the scalar theory of On Learning the Optimal Regularization Parameter hints, then a five-point sweep does the job at comparable cost. The evidence against it is indirect: TFPnP and Self-Tuning Networks both find that dynamic schedules beat the best static setting. That evidence uses ground-truth or labeled validation, though, and does not establish the same benefit under a noisy measurement proxy. The matched-compute ablation settles it. If the static sweep is within noise of our method, we will report that measurement-split cross-validation is sufficient, and that the hypergradient machinery is unnecessary.
Why not just use TFPnP's trained policy? It needs clean images to compute its reward and is trained on a fixed operator family. Both assumptions break exactly where tuning is hardest: new scanners, new masks, and unknown noise. The operator-shift experiments measure how much it loses there. If it loses nothing, our method's value reduces to not needing training data at all.
Why not just use the discrepancy principle? It needs the noise level $\delta$, which is rarely known in clinical MRI or phase retrieval. It also only chooses a stopping point or a scalar. Early Stopping for Deep Image Prior further notes that residuals of high-capacity priors can keep falling after noise fitting begins. We run it with both the true and a perturbed $\delta$ to quantify this sensitivity.
Why not just restrict capacity, as in DIP subspaces? This removes the stopping problem but needs offline pretraining, and it over-smooths fine detail in low-noise regimes, as Deep Image Prior Subspaces acknowledges. The approach is complementary rather than competing: our held-out loss could choose its subspace dimension. We compare against it only in the untrained-prior extension.
Proposed Experiments
The primary testbed is MICCAI CS-MRI at 2x and 4x acceleration, with the BSD-trained denoiser, matching the TFPnP setup so that its 36.11 dB figure is directly comparable. Robustness is tested on the fastMRI single-coil knee data at 4x and 8x (fastMRI). For context only, a fully supervised unrolled network, i-RIM, reports 33.65 dB PSNR / 0.781 SSIM at 4x and 30.56 dB / 0.687 at 8x on the single-coil test set. We do not expect a per-instance method to match it. Instead we report the gap to it and to the oracle schedule. The operator-shift test trains TFPnP at 4x and evaluates both methods at 8x and with a different mask density.
To probe null-space blindness, we add sparse-view and limited-angle CT on the Mayo and Walnut data used by Deep Image Prior Subspaces, which reports 33.76 dB for Sub-DIP with natural-gradient optimization against 32.55 dB for E-DIP. Phase retrieval is also included, since TFPnP reports 40.33 dB against prDeep's 39.70 dB in the high-SNR regime. For self-supervised MRI context, SelfDEQ reports 31.28 dB / 0.903 SSIM at $R=6$ on simulated brain data. Metrics are PSNR, SSIM, wall-clock time and number of forward-operator evaluations relative to one reconstruction, and variance across random splits.
Three outcomes are decisive.
The first is the rank correlation between held-out loss and true PSNR, computed per operator. Where it is weak, gradient tuning on it cannot be trusted, and we expect the limited-angle CT gains to vanish accordingly.
The second is the matched-compute comparison against static held-out cross-validation. It decides whether schedules tuned by gradient matter at all without ground truth.
The third is the shifted-operator comparison against TFPnP. It decides whether removing ground truth buys robustness. A result where we match TFPnP in distribution, beat it under shift, and beat static cross-validation by more than the split variance would establish the claim. Any other pattern tells us precisely which part of the claim fails.
References
- TFPnP: Tuning-free Plug-and-Play Proximal Algorithms with Applications to Inverse Imaging Problems
- Tuning-free Plug-and-Play Proximal Algorithm for Inverse Imaging Problems
- On learning the optimal regularization parameter in inverse problems
- Regularization of inverse problems: deep equilibrium models versus bilevel learning
- Scalable Gradient-Based Tuning of Continuous Regularization Hyperparameters
- Self-Tuning Networks: Bilevel Optimization of Hyperparameters using Structured Best-Response Functions
- Self-Supervised Deep Equilibrium Models for Inverse Problems with Theoretical Guarantees
- Early Stopping for Deep Image Prior
- Image Reconstruction via Deep Image Prior Subspaces
- Optimization for Amortized Inverse Problems
- Convergence analysis of ensemble Kalman inversion: the linear, noisy case
- fastMRI: An Open Dataset and Benchmarks for Accelerated MRI.
- i-RIM applied to the fastMRI challenge