A single image-restoration network that removes haze, rain and low resolution from overhead imagery has to cope with two problems at once. The first is that the three degradations want different things from the same filters: haze removal wants smooth, wide-area corrections, while rain removal and super-resolution want sharp local edges. The second is that remote sensing images come from sensors whose pixels cover anywhere from a few centimeters to tens of meters on the ground. A 3×3 kernel or a fixed dilation rate (the spacing between the pixels a filter looks at) therefore covers a completely different physical area on each sensor. This note argues that these two problems are largely the same problem. Much of the gradient conflict between restoration tasks in a shared network, meaning cases where the update that helps one task hurts another, is caused by a mismatch between the filters' physical footprint and the physical scale of each degradation. If that is true, then conflict between tasks is a measurement that tells us how to set resolution-dependent parameters for a new sensor, using a few gradient evaluations instead of retraining.
The practical goal is a restoration model whose spatial parameters are written in meters rather than pixels. To move it to a new satellite or a new drone altitude, we change only a handful of physical-extent scalars, and we choose them by maximizing agreement between the per-task gradients on a small calibration batch. Today's options look poor by comparison. Multi-weather remote sensing restorers hardcode their scanning patterns, and full fine-tuning is the documented fallback when adapters fail on overhead data.
Core Claim
Our hypothesis is that the cosine similarity between task gradients on shared layers varies systematically with ground sampling distance (GSD, meters of ground per pixel) when spatial parameters are fixed in pixels, and that this dependence largely disappears when they are fixed in meters. We write each dilation or scan stride as $r = \rho / s$, where $s$ is the input GSD and $\rho$ is a learnable physical extent in meters, realized with fractional dilation by bilinear sampling so that $r$ need not be an integer. For a candidate setting $\rho$ at GSD $s$, we define the alignment score
$$A(\rho, s) = \frac{2}{T(T-1)} \sum_{i<j} \cos\big(g_i(\rho, s),\, g_j(\rho, s)\big),$$
where $T$ is the number of restoration tasks and $g_i$ is the gradient of task $i$'s loss with respect to the shared trunk parameters on a calibration batch at GSD $s$. The method claim is that choosing $\rho$ for a new sensor by maximizing $A$ predicts quality after adaptation better than zero-shot validation error does. The reason is that $A$ measures whether the tasks will cooperate during the fine-tuning that follows, not how good the frozen model looks now.
Two lines of prior work make this different from what exists. Gradient-alignment methods manipulate updates to resolve conflict that they take as given. Conflict-Averse Gradient Descent for Multi-task learning (CAGrad) searches, at every step, for the update that most improves the worst-off task while staying within a ball around the average gradient. Understanding SAM's Robustness to Noisy Labels through Gradient Down-weighting masks individual gradient coordinates using the ratio between the sharpness-aware gradient and the plain gradient. Neither has been applied to restoration or remote sensing, and neither asks where the conflict comes from. We use gradient agreement as a diagnostic and a selection criterion rather than as optimizer surgery. That makes the cost a few backward passes at calibration time, not a per-step multi-gradient computation. On the architecture side, Weamba restores haze and rain in remote sensing imagery with a state space model, a linear-time sequence model run over flattened image patches, using fixed 'Z', 'S' and 'W' scan paths and a plain $L_1$ loss. AtrousMamaba fixes its scan dilations at 2, 5, 7 and 9 pixels for change detection. Both supply the backbone we build on, and both leave the scale in pixel units. Our contribution is the physical reparameterization together with the conflict-based rule for setting it.
The motivating evidence is indirect but consistent. Scaling Spatial Hyperparameters Across Image Resolutions shows that fixed kernels approximating spatial quantities break when pixel density changes. Boundary Loss for Remote Sensing Imagery Semantic Segmentation reports that its 2–3 pixel boundary buffer, tuned at one GSD, must be re-tuned by hand at another. Theory on why clean examples' gradients add up while noisy ones cancel, in How Does Label Noise Gradient Descent Improve Generalization in the Low SNR Regime? and Benign Overfitting in Two-layer ReLU Convolutional Neural Networks, supplies the intuition that gradient agreement tracks shared structure. That theory is for classification with label noise, so treating it as a signal for restoration across tasks is our hypothesis, not a result we inherit.
Research Program
The first stage is a measurement study, and it can falsify the whole idea cheaply. We train a shared-trunk restorer with task-specific heads on haze, rain and ×4 super-resolution. The trunk is a Weamba-style state space model, plus a convolutional variant to check that the effect does not depend on the architecture. We train it at one GSD, then log $A$ on held-out batches across a ladder of GSDs. To build the ladder, we take RSSOD imagery from Remote Sensing Image Super-resolution and Object Detection, at roughly 0.05 m GSD, and downsample it by known factors. We then synthesize haze and rain on each rung, so that every degradation's physical scale is controlled. The prediction is that, with pixel-fixed dilations, $A$ falls as GSD departs from the training GSD. A pixel-fixed setting matched to the training GSD would also show this falloff, so the decisive comparison is the second one: with meter-fixed $\rho$, the curve should stay roughly flat.
The second stage trains the physically parameterized model jointly across the GSD ladder, with $s$ known for each batch, and learns $\rho$ per layer. The third stage is new-sensor adaptation. We freeze everything, sweep or optimize $\rho$ on a calibration batch at the target sensor, select by maximum $A$, and then optionally fine-tune briefly. The targets are real data the model never saw: the real satellite haze in RRSHID from Real-World Remote Sensing Image Dehazing: Benchmark and Baseline and the drone rain in UAV-Rain1k. Calibration uses only synthetic degradations applied to the target sensor's own imagery, so no real paired data are needed. One practical risk is that the file gives no per-image altitude or GSD metadata for UAV-Rain1k. If it is absent, we use the nominal sensor GSD and report sensitivity to errors in that value.
Tuning only $\rho$ amounts to an extreme form of parameter-efficient adaptation, with a few scalars per layer. That matters because How to Efficiently Adapt Foundation Models to Remote Sensing for Object Detection? found LoRA collapsing in few-shot remote sensing transfer, which forced full fine-tuning. Their task was detection, not restoration, so this result motivates testing low-rank adapters against scale-only adaptation rather than predicting the outcome.
Baselines
The naive reference point is the pixel-parameterized restorer trained once with an equal-weight sum of task losses and applied to the new sensor unchanged. It shows how much the domain and resolution shift costs before any intervention.
The strongest published real-image baselines are task-specific. On RRSHID, Real-World Remote Sensing Image Dehazing: Benchmark and Baseline reports 24.11 dB PSNR and 0.6680 SSIM for MCAF-Net, a dehazing-only network with fixed 3×3, 5×5 and 7×7 multi-scale branches. On UAV-Rain1k, UAV-Rain1k reports 24.93 PSNR and 0.9155 SSIM for DRSformer, a single-task deraining transformer. Weamba, the closest all-in-one remote sensing competitor, is reported as best on UAV-Rain1k in PSNR, but no reported result for Weamba was found on Lacuna. We will therefore retrain it ourselves on our task mix, at the same data, training-iteration budget and hyperparameter-search budget as our model, starting from the authors' recipe.
Beating the specialists is not the bar, since an all-in-one model often trails them. The bar is to match MCAF-Net and DRSformer within a small margin, which we set at 0.3 dB in advance, while transferring zero-shot to these real sets and beating retrained Weamba under identical conditions.
We also run three optimizer and adapter competitors. CAGrad and PCGrad are run on the pixel-parameterized model, with CAGrad's ball radius tuned on the same budget. CAGrad is the reference for "solve conflict in the optimizer": it reports a 0.20% mean per-task drop against PCGrad's 3.97% on NYU-v2 in Conflict-Averse Gradient Descent for Multi-task learning. LoRA adaptation to the new sensor is matched in calibration data and steps to our scale-only adaptation.
Two ablations isolate the key ingredients. The first compares meter-fixed $\rho$ against pixel-fixed dilations with identical architecture and training. The second compares $\rho$ selected by alignment against $\rho$ selected at random within the same range, and against $\rho$ selected by zero-shot synthetic validation PSNR.
Why not just…?
Why not just resample every image to one canonical GSD? This is the cheapest serious alternative and could genuinely match us at mid-range GSDs. Its known costs are at the extremes. Downsampling fine imagery throws away the detail super-resolution is meant to recover, and the pixel-poverty problem documented for RSSOD, with vehicles a few pixels across, gets worse. Upsampling coarse imagery invents structure that the restorer then treats as real. We run canonical resampling at three choices of canonical GSD across the whole ladder. If it is within noise of our model everywhere, the physical reparameterization is not worth it and we will say so.
Why not just feed GSD to the network as a conditioning input and let it learn? A learned GSD embedding that modulates features is a real threat, because it can in principle learn everything our explicit $\rho$ encodes. It differs in two ways. It cannot change the spatial extent a filter physically reads, only how features are reweighted. It also has no principled way to extrapolate to GSDs outside the training range. We train it on the same ladder and test on held-out GSDs both inside and outside that range. We expect it to tie inside the range and lose outside. If it ties outside as well, our selection rule's value reduces to adaptation efficiency.
Why not just apply CAGrad to a standard all-in-one restorer? If conflict is inherent to the tasks rather than induced by resolution, optimizer-level conflict resolution should recover most of the gap at any GSD, and our hypothesis would be wrong. The first-stage measurement settles this. If pixel-fixed models show conflict that does not depend on GSD and that CAGrad removes, we pivot to that finding. CAGrad also requires a separate backward pass per task at every training step, the overhead its own authors flag. Our approach confines that cost to calibration.
Why not just pick $\rho$ by validation PSNR on synthetic pairs at the new sensor? This competitor uses exactly the same label-free calibration data, so the comparison is fair and it may well win. Our bet is that zero-shot PSNR measures the frozen model, whereas alignment predicts how well fine-tuning will go. We test this by checking which criterion better rank-orders candidate $\rho$ values by their PSNR after brief fine-tuning. If PSNR selection is as good, the alignment score survives only as an analysis tool, not as a method.
Proposed Experiments
The decisive checks come in order of cost. The first is the GSD-versus-alignment curve for pixel-fixed and meter-fixed models on the synthetic RSSOD ladder. A flat meter-fixed curve and a sloped pixel-fixed curve support the core hypothesis; parallel curves refute it. The second is held-out-GSD restoration quality, in PSNR and SSIM per task, against canonical resampling, GSD conditioning and CAGrad. The third is real-sensor transfer to RRSHID, reported overall and by thin, moderate and thick haze, and to UAV-Rain1k, against the MCAF-Net and DRSformer figures quoted above and against retrained Weamba. Here we also measure the Spearman correlation between each selection criterion's ranking of $\rho$ values and the PSNR each value achieves after fine-tuning. The fourth is a downstream check, since restoration that inflates PSNR by over-smoothing can hurt recognition. We run a YOLOv5 detector on RSSOD imagery downsampled ×4 and restored by each method. The reference points are 0.58 mAP for YOLOv5 on the downsampled imagery and 0.711 mAP for MCGR's jointly trained super-resolution and detection, both reported in Remote Sensing Image Super-resolution and Object Detection. Our restorer is not trained with a detection loss, so matching MCGR would be strong evidence that the restoration preserves the edges detection needs. All results are reported over three seeds, with alignment computed on fixed calibration batches so the selection rule is reproducible.
References
- Conflict-Averse Gradient Descent for Multi-task learning
- Understanding SAM's Robustness to Noisy Labels through Gradient Down-weighting
- Weamba: Weather-Degraded Remote Sensing Image Restoration with Multi-Router State Space Model
- AtrousMamaba: An Atrous-Window Scanning Visual State Space Model for Remote Sensing Change Detection
- Training Deep Networks to be Spatially Sensitive
- Boundary Loss for Remote Sensing Imagery Semantic Segmentation
- How Does Label Noise Gradient Descent Improve Generalization in the Low SNR Regime?
- Benign Overfitting in Two-layer ReLU Convolutional Neural Networks
- Remote sensing image super-resolution and object detection: Benchmark and state of the art
- Real-World Remote Sensing Image Dehazing: Benchmark and Baseline
- UAV-Rain1k: A Benchmark for Raindrop Removal from UAV Aerial Imagery
- How to Efficiently Adapt Foundation Models to Remote Sensing for Object Detection?