Null-Space Adapters: Spending Fine-Tuning Capacity Only Where the Sensor Is Blind

When a pretrained image-reconstruction network is moved to a new scanner, a new undersampling pattern or a new coded aperture, part of what it must relearn is already handed to it by physics. These networks recover a full image from a small number of linear measurements, as in accelerated MRI or single-shot hyperspectral cameras. Any image consistent with the new measurements differs from the truth only in the directions the sensor cannot see, which mathematicians call the null space of the measurement operator. Measurements pin down everything else. This proposal argues that lightweight fine-tuning of such networks should be confined, by construction, to that null space. The small trainable add-on modules (adapters) should be allowed to change the reconstruction only where the measurements are silent. The failures that remain under that restriction are then a clean, measurable form of "representation rigidity": the frozen network does not know what plausible content looks like in the blind directions of the new sensor.

The practical claim is that a null-space-projected adapter matches or beats unconstrained adapters, such as low-rank adaptation and output-head adapters, at a fraction of the parameters and with guaranteed measurement fidelity. The scientific claim is that the ceiling such adapters hit can be predicted before any training. The predictor asks how much of the target images' null-space content can be linearly read out from the frozen network's internal features. If both claims hold, they turn representation rigidity from an anecdotal complaint into a quantity that tells a practitioner whether to adapt or retrain.

Background and the gap

Three bodies of work meet here. Each leaves the same seam open.

Compressive sensing recovers signals from far fewer linear measurements $y = \Phi x$ than unknowns, as long as the signal is sparse in some basis. Deep unfolding networks turn the iterations of a recovery algorithm into network layers, alternating a data-consistency step that uses $\Phi$ with a learned denoising step. Examples are DECONET, which unrolls an analysis-sparsity solver with a shared learned operator and proves generalization bounds, and FHDUN, which converges in 8 unrolled phases instead of 20–25. All of these are trained for a fixed $\Phi$. How to warm-start your unfolding network improves their training but likewise never transfers them to a new operator. These papers supply our backbones and expose the gap: nobody has studied what breaks when $\Phi$ changes.

Parameter-efficient fine-tuning (PEFT) freezes a large network and trains a small add-on. Variants include a low-rank weight update (LoRA), the higher-rank Hadamard-product update of ABBA-Adapters, and the parallel residual branch of Res-Tuning. These methods are tuned purely by empirical loss and know nothing about measurement physics. They supply the adapter parameterizations we reuse.

The two closest papers each connect the fields in only one way, and they are what this proposal must clearly differ from. FedPR adapts a frozen fastMRI reconstruction backbone with input-space visual prompts. Its "null space" is an empirical one, computed by SVD of the global prompt to avoid clients overwriting each other, not the physical null space of $\Phi$. The authors themselves report that prompting is bounded by the backbone's feature coverage. SAH-SCI attaches a 0.29M-parameter adapter at the output of frozen hyperspectral snapshot reconstructors. It uses measurement consistency and equivariance as self-supervised losses and a null-space uncertainty term as a regularizer. It still lets the adapter move the image in any direction and only penalizes inconsistency afterward.

Our proposal differs from both in three ways:

  1. The physical null space is a hard architectural constraint on the adapter's output.
  2. The adapter sits inside every unrolled phase, not only at the end.
  3. We measure the residual rigidity and try to predict it.

The rigidity literature motivates the diagnostic half. Few-shot Adaptation of Medical Vision-Language Models finds elaborate prompt learning failing to beat a linear probe when the backbone lacks domain grounding. Path-CLIP finds adapters unable to recover textures the backbone never saw. Both are classification settings where "missing features" cannot be localized. Reconstruction is different: physics tells us exactly which part of the answer the network must invent.

Core Claim

Write the target operator as $\Phi'$. Write its pseudo-inverse as $\Phi'^{+}$, and the projector onto its null space as $P_{\mathcal N} = I - \Phi'^{+}\Phi'$. For undersampled Cartesian MRI this projector is just "zero the acquired k-space lines". For snapshot spectral imaging $\Phi'\Phi'^{\top}$ is diagonal, and for block compressive sensing the projector can be precomputed. In each unrolled phase $t$, a frozen backbone produces $x_t$. The null-space adapter adds a correction:

$$ x_t \leftarrow x_t + P_{\mathcal N}\, g_{\theta,t}(x_t), $$

Here $g_{\theta,t}$ is a small trainable module with parameters $\theta$. It is a LoRA or ABBA-style update to the frozen denoiser's weights, rerouted as a residual branch. The correction is invisible to the sensor ($\Phi' P_{\mathcal N} = 0$), so the adapter cannot degrade agreement with the measurements. All of its capacity goes to filling blind directions.

The diagnostic half defines a rigidity score $\rho$. On a held-out set of target images $x^\star$, fit the best linear map from the frozen network's intermediate features to the null-space component $P_{\mathcal N} x^\star$. Then report the fraction of that component's energy left unexplained. The hypothesis is that $\rho$, computed with no adapter training at all, predicts the error floor that any adapter of fixed budget reaches. Above some threshold, only unfreezing the backbone helps.

Research Program

The work has three parts.

Operator-shift testbeds. We build controlled shifts in which the backbone is trained on one $\Phi$ and deployed on another, and the shift's size can be dialed:

  • For MRI, change the acceleration factor and the mask family (equispaced to random Cartesian) on fastMRI, and change the anatomy from knee to brain.
  • For block image compressed sensing, change the sampling ratio and the random Gaussian matrix for an FHDUN-style or DECONET backbone.
  • For hyperspectral snapshot imaging, change the coded aperture on ICVL, which matches SAH-SCI's setting.

Crossing operator shift with content shift (anatomy, scene type) is what separates the two error sources. Operator shift alone should be largely fixed by data consistency. Content shift is where null-space rigidity should appear.

The adapter. The null-space adapter is implemented with three interchangeable inner parameterizations so the projection can be separated from expressivity: LoRA, ABBA's Hadamard product, and a Res-Tuning-style bypass that avoids backpropagating through the frozen backbone.

One honest difficulty shapes the training design. A measurement-consistency loss gives exactly zero gradient to a null-space-only correction, so purely measurement-supervised test-time adaptation cannot train it. We therefore train two ways. The first uses a few paired target images (1, 10 and 100 volumes or images). The second is self-supervised with rotation and flip equivariance, as in SAH-SCI. Transformations move null-space content into measured directions, which gives the adapter a signal.

Rigidity analysis. We compute $\rho$ per layer and per shift, then test whether it rank-orders the final adapter error across shifts. Because $\rho$ can be computed for every layer, we can also ask whether rigidity concentrates in early or late unrolled phases. That answer would tell practitioners where to place adapters.

Baselines

The naive reference is the frozen source network applied to target data with the source operator assumptions untouched. A stronger "free" reference is the frozen network with $\Phi'$ substituted into its data-consistency steps, which costs no parameters. Every adapter must beat the second, not the first.

The strongest existing adaptation method in this exact regime is SAH-SCI. It reports PSNR gains of up to 5.87 dB and SSIM gains of 0.086 on ICVL with 0.29M tuned parameters (SAH-SCI). We rerun it on our shifts with its own losses at a matched parameter count and matched number of target samples, and our method must exceed its PSNR at equal or smaller budget. FedPR-style input prompting is the second physics-domain competitor. It reports more than 4.5 dB PSNR gain over federated averaging baselines with 0.11M transmitted parameters (FedPR). We run it non-federated, on a single site, with the same backbone. For reference on the backbone side, FHDUN reports an average of +0.36 dB over MADUN on Urban100 (FHDUN). The fastMRI paper reports 40–50% relative NMSE improvement of learned models over total-variation reconstruction (fastMRI). Classical total variation is included as the no-learning floor.

Unconstrained LoRA and ABBA inserted into the same unrolled denoisers are the direct PEFT competitors. We found no reported Lacuna results for LoRA or ABBA on compressed-sensing reconstruction, so they are run fresh with an identical hyperparameter search budget.

Full fine-tuning is the upper reference. Our method beats it if it matches full fine-tuning within 0.3 dB at fewer than 2% of the parameters, or if it beats full fine-tuning in the 1- and 10-sample regimes, where overfitting should hurt full tuning.

The key ablation keeps everything identical but removes $P_{\mathcal N}$. A second ablation replaces it with a projector onto a random subspace of the same dimension. This tells us whether the physical null space matters or merely the restriction of output dimension.

All methods share the backbone, the target samples, the number of optimizer steps, and a 20-trial learning-rate and rank search. Results are reported as PSNR, SSIM and NMSE with three seeds.

Why not just…?

Why not just swap in the new operator and skip adaptation entirely? Unrolled networks already call $\Phi$ explicitly, so substituting $\Phi'$ may close most of the gap. This is a real threat, and it is why operator-swap is a mandatory baseline. Our expectation is that it suffices for pure operator shift but not when content also shifts, because the frozen denoiser fills blind directions with source-domain textures. If operator-swap alone closes the content-shifted gap too, the adapter half of the proposal fails. The rigidity analysis would still show this cleanly, through low $\rho$ on all shifts.

Why not just use unconstrained LoRA inside each phase, since data consistency will correct its errors anyway? Interleaved data-consistency steps do partly undo inconsistent corrections. The projection may therefore buy little accuracy, and its value might be only in parameter efficiency and guaranteed fidelity. The "projection removed" ablation settles this directly. If the two match at equal budget, we will report the projection as a fidelity guarantee rather than an accuracy gain. The ABBA evidence, 66.26% versus LoRA's 61.94% on GSM8K (ABBA-Adapters), suggests low rank may also be binding here, which is why both parameterizations are tested.

Why not just attach an output adapter as SAH-SCI does? It is simpler and already self-supervised. But it sees only the final image after every phase has committed to source-domain priors, so it must undo accumulated artifacts post hoc. Running our per-phase adapter against a single null-space-projected output adapter at equal parameters isolates placement from projection.

Why not just fully fine-tune? With enough target data it may win. Its known costs, documented in Path-CLIP and FedPR, are forgetting on small data and a per-site model copy. Our sample-count sweep shows where the crossover lies. The $\rho$ score is meant to predict that crossover in advance.

Decisive Checks

Three outcomes decide the proposal.

First, on content-plus-operator shifts in fastMRI and ICVL, the null-space adapter must beat operator-swap by a clear margin (at least 1 dB) and match or beat SAH-SCI and unconstrained LoRA at equal parameters. Its measurement residual should stay at the backbone's level by construction.

Second, the projection ablation must show that the physical null space beats a random subspace of equal dimension. Otherwise the mechanism is merely a bottleneck.

Third, across at least ten shifts, $\rho$ computed before training must correlate with the final adapter error, with a Spearman rank correlation above 0.7. Shifts with high $\rho$ should also be exactly those where full fine-tuning pulls ahead. Failure of the third check would leave a useful adapter but no measurable account of rigidity. Success would give practitioners a cheap test of whether a frozen reconstruction backbone can be adapted to a new sensor at all.