Acquisition-Conditioned Residual Denoising for Label-Scarce Satellite Imagery

Satellite images arrive with noise that changes from one acquisition to the next. Each pass has its own band-dependent sensor noise, detector striping, and residual haze. A denoiser trained once and frozen sees only a small patch at a time, so it has to guess which noise process it is facing. We propose to remove that guess. A small network reads a few hundred unlabeled patches from the same acquisition and summarizes the noise they share. A second network, called a hypernetwork (a network that outputs the weights of another network), turns that summary into a compact set of weights for a residual denoiser. A residual denoiser predicts the noise and subtracts it, rather than predicting the clean image directly. Adapting to a new acquisition then takes one forward pass, with no labels and no gradient steps.

The setting where this should matter most is the one remote sensing actually faces. Labels are scarce, so downstream classifiers are trained with a few hundred annotated patches on top of a frozen self-supervised encoder. Under those conditions, corrupted inputs hurt more than they would with abundant labels, and cleaning them well should recover much of the loss. Labels are also too scarce to supervise the denoiser with a downstream task loss. We therefore preserve meaning with a label-free term: features from a frozen self-supervised encoder should be the same for the cleaned image and the clean image. The core claim is that pooling noise statistics across an entire acquisition, and turning that summary into denoiser weights, beats a single "blind" denoiser of equal size. We expect the advantage to be largest for structured, sensor-specific noise and in label-scarce downstream transfer. We also predict that the advantage shrinks as labels grow, in line with the classification–distortion tradeoff.

Core Claim and What Is New

Three pieces of prior work set the stage, and none of them does this. ADRN supplies the restoration method. It predicts the residual noise cube of hyperspectral imagery and adds a zero-mean penalty on that residual, which prevents global brightness shifts. It reports 37.72 dB mean PSNR on Washington DC Mall against 35.40 dB for NG-Meet. Its weights are fixed, though, and it is evaluated on one scene with no measurement of downstream utility. HyperFedZero and ODPFL-HN supply the adaptation method. Both generate weights for unseen clients from a summary of their unlabeled data, but only for classification under synthetic label-skew splits of CIFAR-like data. MT-Hyper is the closest remote-sensing precedent. It generates per-lake regressor weights from lake metadata and finds that plain multi-task models get worse when metadata is concatenated. That work is tabular, assumes clean reflectance, and names zero-shot adaptation as open.

Our proposal differs from all three in what is conditioned and on what. We condition a pixel-level residual noise estimator, not a classifier. The conditioning signal is the unlabeled imagery of one acquisition, not client identity or metadata. The open problem in the zero-shot personalization line is generalizing to physical sensor shifts rather than label-skew partitions, and this is a direct test of it.

Research Program

We build three parts. The first is an acquisition encoder: a permutation-invariant set network in the style of ODPFL-HN. It reads high-pass residuals of $N$ unlabeled patches from one acquisition and outputs a descriptor $z$. The second is a hypernetwork that maps $z$ to low-rank weight offsets for the last blocks of a shared residual denoising network. We generate these offsets in chunks, following HyperFedZero, so that memory stays small. The third is the residual denoiser $f$ itself, which outputs noise $\hat n = f(y; \theta_0 + \Delta\theta(z))$ and the restored image $\hat x = y - \hat n$. Here $y$ is the observed image, $\theta_0$ are the shared weights, and $\Delta\theta(z)$ are the generated offsets.

Training uses groups of patches that share one sampled noise process, so each group plays the role of a simulated acquisition. We start from sen2cor-corrected Sentinel-2 patches in BigEarthNet v2 and inject physically motivated degradations. These are per-band Gaussian and signal-dependent noise, column striping with acquisition-specific gains, spatially correlated noise, and low-frequency haze. The degradation parameters are held fixed within a group and resampled across groups. The loss has four parts:

$$\mathcal{L} = \lVert \hat x - x \rVert_1 + \lambda_1 \lVert \overline{\hat n} \rVert^2 + \lambda_2 \lVert \phi(\hat x) - \phi(x) \rVert^2 + \lambda_3 \mathcal{B}(z).$$

In this loss, $x$ is the clean patch and $\overline{\hat n}$ is the per-band mean of the predicted residual. The term $\phi$ is a frozen encoder pretrained with Seasonal Contrast (SeCo), and $\mathcal{B}$ is HyperFedZero's balancing penalty, which spreads descriptors apart and prevents the encoder from collapsing to one constant summary. The feature term replaces the supervised Dice regularizer that SINCO adds to its pixel loss. SINCO's term needs a trained segmenter and labels, which we do not have. The SeCo encoder carries meaning learned from revisit pairs without annotation.

The weights $\lambda_i$ are chosen on held-out simulated acquisitions under a fixed search budget. We then report how sensitive results are to each weight, because fixed loss weights are a known source of instability.

The analysis then asks four questions. Does the descriptor $z$ track the true noise parameters, as measured by linear probes from $z$ to striping gain and noise level? How does quality change with the number of pooled patches $N$? Where does residual estimation wrongly subtract real high-frequency content, such as shorelines and field furrows, which is the known failure mode of residual estimators? And how does downstream gain change with the fraction of labels?

Engaging the Classification–Distortion–Perception Result

Liu et al. prove that when the degradation is known, restoration cannot lower the best achievable (Bayes) classification error below that of the degraded signal. Taken literally, this suggests skipping restoration and training on noisy data. We accept the theorem and argue it does not apply to the regime we target. It bounds the best possible classifier, while a linear probe trained on 1% of labels is far from that bound. In our setting the denoiser is a way of transferring knowledge learned from abundant unlabeled and simulated data. This yields a falsifiable prediction: the downstream advantage of denoising over training directly on noisy inputs should shrink toward zero as the label fraction rises to 100%. If the advantage is already absent at 1% labels, the downstream half of the proposal fails.

Baselines

The naive reference point is to feed the degraded patches straight into the frozen SeCo encoder and train a linear probe on them. We do this at 1%, 10%, and 100% of BigEarthNet v2 labels. We also run the same pipeline on clean inputs, which serves as an upper bound. For context, SeCo reports 87.81% mean average precision (mAP) on clean original BigEarthNet and shows that 1% of labels with SeCo beats ImageNet initialization with full labels. BigEarthNet v2 reports a macro average precision of 70.93% for a fused Sentinel-1 and Sentinel-2 ResNet-101. Those numbers come from different protocols, so we use them as sanity anchors and not as targets.

The strongest existing restoration competitor is a static, blind ADRN-style residual denoiser with the zero-mean loss. It is trained on the same mixture of simulated acquisitions as our model, and we enlarge it until its parameter count matches our shared network plus hypernetwork. It is the strongest competitor because blind denoisers trained on mixed noise levels often handle variation well, and nothing in the file rules that out. Beating it means two things. First, higher PSNR and lower spectral angle on held-out acquisitions with unseen noise-parameter combinations, under the geographically disjoint Inner–Outer split. Second, a downstream mAP gain at 1% labels that holds across three seeds. On Washington DC Mall we use ADRN's own Gaussian protocol. There the target is to match its reported 37.72 dB mean PSNR on its home benchmark, using our reimplementation, while showing gains on the striping and correlated-noise settings that ADRN did not test.

Three ablations isolate the key ingredient. The first computes the descriptor from the single patch being denoised rather than from a pool of $N$ patches, which tests whether acquisition-level pooling is what helps. The second replaces generated weights with feature-wise linear modulation (FiLM), a per-channel scale and shift computed from the same pooled descriptor, which tests whether full weight generation is necessary. The third predicts the clean image directly instead of the residual, keeping everything else fixed. As a compute-heavy reference, we also run per-acquisition test-time fine-tuning of the static denoiser using a self-supervised residual likelihood objective in the spirit of Noise2Info. We report its quality alongside its wall-clock time and memory. All methods get the same simulated data, the same tuning budget, and the same frozen downstream encoder.

Why not just…?

Why not just train one bigger blind denoiser? This is the main threat, and it is why the parameter-matched blind baseline and the single-patch ablation exist. Our argument is about information, not capacity. A 64-pixel patch often cannot tell low-gain striping apart from real linear features such as roads and field edges, whereas hundreds of patches from the same pass can. If the blind model matches ours on striping and correlated noise, the pooling hypothesis is wrong, and we would report that.

Why not just train the classifier on noisy data, as the classification–distortion theorem suggests? We include this directly, both with and without noise augmentation during probe training. The theorem constrains the limit of abundant labels and makes no claim about the label-scarce regime. Our label-fraction sweep tests exactly where the two approaches meet.

Why not just condition with FiLM instead of a hypernetwork? MT-Hyper's finding that metadata concatenation hurt plain multi-task models is suggestive, but it comes from tabular regression and does not settle the question for imagery. FiLM is a real contender. If it matches generated low-rank offsets, we adopt FiLM as the conditioning mechanism, because the claim we care about is acquisition-level conditioning, not hypernetworks as such.

Why not just fine-tune per scene, with MAML or adapters? METEOR shows that gradient-based adaptation works for Earth observation, reporting 90.8% marine-debris accuracy from 5 images. However, it needs labeled support examples and an inner loop of gradient steps at inference. Our self-supervised test-time fine-tuning baseline measures the quality ceiling that such adaptation reaches. The open question is whether a single forward pass gets close enough to that ceiling to justify running at a fraction of the cost.

Proposed Experiments and Decisive Checks

We train on simulated acquisitions built from the Inner regions of BigEarthNet v2 and test on Outer regions with held-out degradation settings. We report PSNR and SSIM per band, spectral angle, and NDVI error, the last because it measures whether physical reflectance values survive cleaning. For downstream evaluation we use BigEarthNet v2 multi-label mAP and transfer to EuroSAT at matched label fractions. We also run a qualitative and statistical check on the real cloud- and shadow-affected patches that BigEarthNet v2 flags. Washington DC Mall checks the hyperspectral case against ADRN's protocol.

The decisive outcome has three parts, and all three must hold. The pooled-descriptor model must beat both the parameter-matched blind denoiser and the single-patch ablation on structured noise. Its downstream advantage over training on noisy inputs must be clear at 1% labels and shrink at 100%. And its descriptor must linearly predict the injected noise parameters. If the first part fails, the idea fails.