Satellite and aerial images are rarely damaged everywhere at once. A cloud covers one corner of a scene, haze softens a valley, or a sensor stripe crosses a few rows, and the rest of the tile is still clean. Vision Transformers (models that cut an image into small square patches and let every patch "attend to", or pull information from, every other patch) handle this badly. A degraded patch competes for attention on equal terms with a clean one, so cloud texture leaks into the features of the cars, ships, and fields the model is meant to recognize. Current practice either assumes inputs are clean or throws away whole tiles that look bad. We propose a different route. A small head predicts, for every patch token, how trustworthy that patch is. This prediction is a spatial quality map at exactly the resolution the Transformer already works at. The map then down-weights degraded patches as sources of information inside self-attention, while still letting them be looked at as targets.
The core claim is narrow and testable. On a remote-sensing backbone that is already adapted to overhead viewpoints, an explicit token-aligned quality map used as an attention bias recovers more accuracy on partially degraded scenes than giving the same backbone the same degraded training data and the same parameter budget without the map. It also does this where no sensor-provided cloud mask exists. The quality head is trained on degradations composited onto clean tiles, so its ground truth is exact by construction and not borrowed from a classical quality metric.
What exists and what is missing
Three bodies of work motivate the proposal without covering it. Spatial quality mapping for natural photos shows that a network can predict where an image is degraded without seeing a clean reference. BPSQM trains a U-Net to imitate full-reference maps such as SSIM. TempQT pairs a Transformer that predicts a pixel error map with a learned token that weights patches by perceptual importance. Both output a single score for a ground-level photo; neither feeds the map back into a downstream recognition model. BPSQM in particular inherits whatever its classical teacher metric misses. SPAIR supplies the closest mechanism. It predicts a binary degradation mask and restricts non-local attention so that degraded pixels draw context only from clean regions. However, it works on pixels with sparse convolutions, targets rain and blur in ground photos, and aims at restoration rather than recognition.
Remote-sensing foundation models close the viewpoint gap but treat every patch as equally reliable. The plain ViT with rotated varied-size window attention (RVSA) pre-trains with masked autoencoding on MillionAID and adds attention windows that rotate and resize. It reports 81.24% mAP on DOTA-V1.0 with roughly half the memory of full attention. GFM continually pre-trains a Swin student by distilling from an ImageNet-22k teacher, with the same weight on every token. On the data side, ScoreRS shows that filtering remote-sensing image–text pairs by a learned quality score helps: training on the top 30% gives +7% classification on EuroSAT/fMoW. Its score is one scalar per pair, so a tile that is 10% cloud is either kept whole or lost whole.
The proposal therefore reuses three things: SPAIR's idea of masking attention by degradation, TempQT's idea of a quality signal living in token space, and RVSA's backbone. It differs from all of them in what it does with the map. The map is a continuous, per-token reliability estimate injected into a remote-sensing Transformer at inference, so that recognition on overhead imagery degrades gracefully. This is the gap identified across GFM, RVSA, and ScoreRS.
Core mechanism
Let $q_j \in (0,1]$ be the predicted quality of patch token $j$, produced by a lightweight head reading early-layer tokens of the frozen backbone. Let $Q_i, K_j$ be the query of token $i$ and key of token $j$, with head dimension $d$. In selected layers, attention becomes
$$ A_{ij} = \mathrm{softmax}_j\!\left(\frac{Q_i K_j^\top}{\sqrt{d}} + \lambda \log q_j\right), $$
where $\lambda \ge 0$ is a learned per-layer scalar. A patch judged fully degraded ($q_j \to 0$) stops contributing information to others. It still receives information, so a car half under haze can borrow context from clear road around it. When $\lambda = 0$ the backbone is unchanged, which makes the ablation clean. Inside RVSA's rotated windows the bias applies within each window, so the mechanism keeps RVSA's orientation handling and memory savings.
The head is trained with a per-patch regression target equal to the fraction of the patch's pixels that were degraded, together with the downstream task loss. Training data are clean DOTA, DIOR-R and GeoPile tiles with composited degradations. These are real cloud and shadow cut-outs, synthetic haze with spatially varying transmission, sensor striping, saturated blocks, and local compression. Since we placed the damage ourselves, the target is exact, which removes BPSQM's dependence on a teacher metric. With 16-pixel patches on a 1024-pixel crop the map is a 64×64 grid. That is finer than the 20×20 coordinate grid of Grounding-IQA, whose cells are larger than most DOTA objects.
Research program
The first stage builds the degradation compositing pipeline and the quality head on a frozen RVSA backbone, using the official RVSA code. We measure how well the head localizes degradation, as per-patch correlation and IoU at a fixed threshold, both on held-out composites and on real cloudy scenes.
The second stage trains the attention bias jointly with the detection or classification head and measures accuracy as a function of degraded-area fraction. The key output is a curve, not a single number. We expect the advantage to grow with coverage until degraded area dominates and nothing is left to recover.
The third stage tests real rather than synthetic degradation. It draws on the revisit structure that SeCo exploits. For Sentinel-2 footprints with a clear acquisition and a partially clouded one a few weeks apart, land-cover labels from the clear date transfer to the cloudy date. This gives paired evaluation in which only the atmosphere changed. Following BigEarthNet v2, all splits are geographic. Cloud and haze textures used for compositing come from regions disjoint from the evaluation regions, so the head cannot memorize location-specific weather.
Baselines
The do-nothing reference is the frozen RVSA backbone with its task head fine-tuned on clean data only, then evaluated on degraded inputs. For clean DOTA-V1.0 the paper reports 81.24% mAP (RVSA). No reported result was found on Lacuna for RVSA, or any model in the file, on partially degraded DOTA, DIOR-R, or cloudy Sentinel-2. So the first deliverable is establishing how far this baseline actually falls. For land cover, the reference number is the 70.93% macro AP of a ResNet-101 on fused Sentinel-1 and Sentinel-2 under the Inner-Outer split (BigEarthNet v2). We re-run RVSA on our clear/cloudy paired subset rather than compare across subsets.
The strongest competitor is degradation-augmented adaptation without a map. The same backbone receives the identical composited training stream and an adapter with the same number of trainable parameters as our quality head plus bias. This is the method a skeptic would actually deploy, and it isolates whether the explicit map adds anything beyond exposure to degraded data. We run both with identical data order, epochs, and hyperparameter search budget (the same number of trials over learning rate and adapter size), and report mean and spread over three seeds. Beating it means a gain larger than seed spread at medium coverage (30–60% of the tile degraded) on both DOTA and the real cloudy pairs, with no loss on clean inputs.
Two further baselines bracket the method. First, a sensor-mask oracle substitutes the cloud and shadow classes from Sentinel-2's atmospheric-correction product for $q_j$. This is an upper reference where such masks exist. Second, a restore-then-recognize pipeline runs a SPAIR-style mask-guided restorer (SPAIR) trained on the same composites, then the clean-trained detector. Two ablations isolate the key ingredient. The first trains the quality head but fixes $\lambda = 0$, so the map exists but does not touch attention. The second replaces the predicted map with a random map of matched sparsity, which tests whether any attention sparsification would help.
Why not just…?
Why not just augment training with degraded images and let attention learn to ignore clouds? This is the real threat, and the proposal is built around it. Attention can in principle learn to suppress cloud keys on its own. The argument for an explicit map is that supervising "where is the damage" with exact per-patch targets is an easier learning problem than discovering it indirectly from a detection loss, especially with limited labels. The equal-budget comparison above settles it. If augmentation alone matches us on real cloudy pairs, the honest conclusion is that the map's value reduces to its by-product: an interpretable, patch-level quality output for curation. We would report it that way.
Why not just use the cloud mask that ships with the satellite product? Where such a mask exists, it is a strong signal, and we use it as an oracle reference. But it exists for Sentinel-2 after atmospheric correction. It does not exist for the Google Earth and aerial RGB imagery behind DOTA, DIOR and MillionAID. It also covers clouds and shadows but not haze gradients, striping, saturation, or compression. Our claim is restricted to closing most of the gap to the oracle where the oracle exists, and helping where it does not.
Why not just discard bad tiles, as data curation does? ScoreRS shows that scalar filtering helps when bad pairs are bad throughout, and its CLIP-score comparison shows coarse global scores miss subtle problems. At inference there is no option to discard the tile you were asked to analyze. In training, discarding a tile with a cloud in one corner loses the clean majority. The coverage curve directly shows how much accuracy is recoverable from tiles a scalar filter would reject.
Why not just restore the image first and then run the normal model? SPAIR itself shows that uniform restoration damages clean pixels, which is why it restricts processing to masked regions. Even spatially adaptive restoration under thick cloud must invent the terrain underneath, and a detector cannot tell invented vehicles from real ones. The restore-then-recognize baseline tests this, and we additionally count false positives inside degraded regions, where hallucinated content should show up.
Decisive checks
The proposal succeeds if three things hold together. The quality-gated model beats the equal-budget augmentation baseline on the real clear/cloudy Sentinel-2 pairs, not only on synthetic composites. Its clean-input accuracy stays within seed noise of the do-nothing RVSA baseline. The $\lambda = 0$ ablation loses most of the gain, which shows the benefit comes from routing attention and not from the extra head's training signal. A further diagnostic checks that learned $\lambda$ is largest in middle layers and that attention mass on degraded keys actually drops. If the gain appears only on synthetic composites, the result says the compositing pipeline is unrealistic, and the next step is better degradation models, not architecture.
References
- Blind Predicting Similar Quality Map for Image Quality Assessment
- Blind Image Quality Assessment via Transformer Predicted Error Map and Perceptual Quality Token
- Spatially-Adaptive Image Restoration using Distortion-Guided Networks
- Advancing Plain Vision Transformer Towards Remote Sensing Foundation Model
- Towards Geospatial Foundation Models via Continual Pretraining
- Quality-Driven Curation of Remote Sensing Vision-Language Data via Learned Scoring Models
- Grounding-IQA: Grounding Multimodal Language Model for Image Quality Assessment
- Seasonal Contrast: Unsupervised Pre-Training from Uncurated Remote Sensing Data
- BigEarthNet v2