Leave-Region-Out Momentum Prototypes for Denoising Pseudo-Labels in Satellite Segmentation

Land-cover maps are drawn by segmentation models, which assign a class such as road, building or field to every pixel of an aerial or satellite image. These models lose accuracy sharply when moved to a region they were not trained on. The usual remedy is self-training, also called pseudo-labeling. The model labels the new, unlabeled region itself and then retrains on its own guesses. The trouble is that its wrong guesses are not scattered at random. Neighbouring places share soils, roof materials and sensor conditions, so a model that misreads one tin-roofed district tends to misread the whole district. Self-training then reinforces that regional mistake.

We propose checking each pseudo-label against evidence from somewhere else. The model keeps a slowly updated memory of what each class looks like, and the memory is split by geographic region. A pixel's pseudo-label is trusted only if class summaries built from other regions agree with it. This borrows spatial cross-validation from geostatistics, where one tests on held-out blocks of space rather than on random pixels. Here the idea is moved inside the training loop as a noise filter.

Core Claim

Our claim is specific and falsifiable. Under geographic domain shift, pseudo-label errors in dense segmentation are spatially clustered beyond what class composition alone explains. Because of this clustering, the standard momentum-contrast memory is itself corrupted. That memory is a first-in-first-out queue of features produced by a slowly updated "key" encoder, an exponential moving average of the trained model. A queue filled from consecutive batches is dominated by whatever region was recently sampled, so its "class" summaries quietly become regional summaries. Features that cluster around them end up agreeing with the very errors they should catch.

We propose a geographically stratified momentum memory to fix this. It stores one prototype per class, per geographic cell, and per feature-pyramid scale. For a target pixel $i$ in cell $g$, we compute a leave-region-out prototype $p_c^{(-g)}$. This is the class-$c$ prototype averaged over cells outside a spatial buffer around $g$. The pixel's pseudo-label weight is then

$$w_i = \mathrm{softmax}_c\left(\frac{\mathrm{sim}(z_i, p_c^{(-g)})}{\tau}\right)\Big[\hat y_i\Big],$$

where $z_i$ is the pixel's key-encoder feature, $\hat y_i$ is the teacher's predicted class, $\mathrm{sim}$ is cosine similarity, and $\tau$ is a temperature. The weight is high only when out-of-region evidence supports the teacher. The same stratification is used to draw negatives for a pixel-level contrastive loss, so negatives span regions rather than echoing the current batch's neighbourhood.

Relation to Prior Work

Two papers motivate the gap. LoveDA benchmarks urban–rural adaptation across three Chinese cities. It shows that self-training beats adversarial alignment on this shift but stays well below the upper bound obtained by training on labeled target data, and that pseudo-labels starve rare classes. STAR detects spatial clusters of prediction errors with likelihood scanning and branches the network per region. That is direct evidence that errors cluster in space. STAR, however, uses labels and a costly scan, and does nothing about pseudo-labels on unlabeled targets.

Several papers supply the method. Towards Discriminative Representation Learning for Unsupervised Person Re-identification uses a momentum encoder and a dynamic queue of cluster-level entries to keep noisy clustering pseudo-labels from corrupting identity features. On Market→Duke, that component alone lifts mAP from 53.7% to 59.6%. It works on whole-image identities, where there is no spatial structure to exploit. Prototypical momentum contrast shows that clustering the memory into prototypes reduces class collision, the error of pushing same-class samples apart. We make those prototypes region-indexed and scale-indexed. Self-Ensembling Attention Networks uses a moving-average teacher and a single spatial mask to down-weight inconsistent regions. Our weight differs in two ways. It is class-specific rather than a single mask. It is also derived from out-of-region evidence rather than from teacher–student disagreement, which a regionally biased teacher and student can share.

Two papers constrain the design. Scale Aware Adaptation shows that scale matters. It adds a scale attention module and a scale discriminator, and reports that naive bilinear resampling of target images hurts. This is why the memory is kept per pyramid level: roads are recognizable at fine scales and fields at coarse ones. GeoMultiTaskNet shows that coordinates are a useful training signal. It regresses coordinates as an auxiliary task. We use coordinates differently, to index memory and hold regions out, and never as a prediction target.

Pseudo-Label Validation selects checkpoints without target labels by scoring them against an auxiliary model's soft predictions. We adopt a spatially blocked version of it as the shared protocol for model selection. WILDS 2.0 warns that adaptation methods often fail to beat plain training once tuning is fair. This is why we tune every method under the same label-free protocol.

Research Program

The system is a momentum teacher–student segmentation network. The student is trained by gradient descent. The teacher is its moving average and produces pseudo-labels.

The memory holds class × cell × scale prototypes. Each prototype is updated from teacher features of pixels whose pseudo-label weight exceeds a threshold. Cells come from image coordinates where they exist; FLAIR has them, as GeoMultiTaskNet's use shows. Where they do not, we use city or scene identity. We also test a coordinate-free fallback in which cells are clusters of image-level embeddings. This fallback tells us whether true geography matters or only a proxy for visual regime.

At each pyramid scale, the final pseudo-label weight is the leave-region-out agreement defined above. The per-scale weights are combined using the teacher's existing scale-attention weights. No new attention module is designed; multi-scale attention appears only as the selector of which scale's evidence to trust. Training minimizes weighted cross-entropy on target pixels plus the stratified contrastive loss, with class-balanced prototype updates to counter the rare-class starvation LoveDA documents.

Alongside the method, we run a measurement study. For each baseline, we map pseudo-label errors on a held-out labeled audit subset of the target. We compute their spatial autocorrelation after conditioning on class. We also track how regionally concentrated a standard FIFO queue becomes over training. This study is the premise check for the whole proposal.

Baselines

The naive reference point is the source-only model: the same backbone trained on labeled source data and applied directly to the target. It establishes how large the geographic gap is on each split.

The strongest simple competitor is class-balanced self-training with a momentum teacher. This is a moving-average teacher producing hard pseudo-labels above a confidence threshold, with per-class thresholds in the style of CBST. The file shows CBST at 41.32% mIoU on both LoveDA Rural→Urban and Urban→Rural, against an upper bound of 45.67% and 46.79%. The file lists the same 41.32% for both directions, which we will re-verify in our own reproduction. IAST reaches 40.48% on Rural→Urban. Beating this baseline means closing a meaningful fraction of the remaining gap to the upper bound, which we set at one mIoU point or more averaged over three seeds, with gains concentrated on rare and slender classes. No reported result was found on Lacuna for a mean-teacher baseline on LoveDA, so we run it ourselves.

On FLAIR, the strongest reported method is GeoMultiTaskNet at 47.22% mIoU. We rerun it with the same backbone, and also stack it with our method, since the two use coordinates differently. The file says GeoMultiTaskNet beat DAFormer on FLAIR but gives no DAFormer number; no reported result was found on Lacuna for DAFormer on FLAIR. For the scale-shift split, the reference is Scale Aware Adaptation at 47.66% on Potsdam→Vaihingen and 27.35% on DeepGlobe→Vaihingen.

The key ablation replaces the stratified memory with a standard FIFO momentum queue of pixel prototypes, while keeping the same agreement weighting but computed in-region. A second ablation keeps the memory stratified but drops the leave-region-out step, using all cells including $g$. A third collapses the memory to a single scale. Together these isolate the three design choices.

All methods share the same backbone, training schedule, augmentations and GPU budget. Each receives the same number of hyperparameter trials, selected by the spatially blocked pseudo-label validation criterion. For transparency we also report selection by target labels, the oracle choice.

Why not just…?

Why not just use a mean teacher with confidence thresholds? This is the real threat. Moving-average teachers already smooth pseudo-labels, and Self-Ensembling Attention Networks shows large gains from them on driving scenes. Our argument is that teacher and student share the same regional bias, so their agreement cannot catch a region-wide error. That argument is only true if errors are spatially clustered. The measurement study tests this directly. If class-conditional error autocorrelation is negligible, the mean teacher should match us and we will report that.

Why not just average the teacher's predictions over several input scales? Multi-scale test-time averaging is cheap and addresses the scale-dependence of noise without any memory. However, Scale Aware Adaptation found that resampling target imagery underperformed, at 40.74% against 43.09% for standard adaptation. Averaging over scales also leaves regional bias intact. We include multi-scale-averaged pseudo-labels as a baseline. If it matches the full method, the geographic component adds nothing.

Why not just use an ordinary momentum prototype queue, as in re-identification work? Isobe et al. show that such queues help against clustering noise. Our ablation replaces stratification with FIFO and keeps everything else fixed. The prediction is that the gap widens on the splits with the strongest measured error clustering. If it does not, stratification is unnecessary.

Why not just train on more geographically diverse source data? Overcoming the Geographic Generalization Gap finds that source diversity beats proximity. That is complementary rather than competing, because an unlabeled target region still needs adaptation. We test it by adding source cities in LoveDA and checking whether our gains persist on top of the more diverse source.

Proposed Experiments

The benchmarks are LoveDA in both directions (Rural→Urban and Urban→Rural), FLAIR (coordinate-indexed cells), and Potsdam→Vaihingen (cells are image tiles, to probe scale shift). Metrics are mIoU and per-class IoU, with results stratified by object size so the effect on slender classes is visible. We also report the precision and recall of the weighted pseudo-labels on a labeled audit subset, and training overhead in time and memory.

The first decisive check is the premise. After conditioning on class, is pseudo-label error spatially autocorrelated on these splits? And does a FIFO queue's regional concentration predict prototype drift? If not, the proposal stops there, and the negative result is reported as a finding about remote-sensing adaptation.

The second check is causal. At equal pseudo-label coverage, leave-region-out weighting should yield higher pseudo-label precision than in-region weighting and than teacher confidence alone. The gap should grow with the measured autocorrelation across splits.

The third check concerns geography versus proxy. Coordinate-indexed cells should outperform embedding-cluster cells on FLAIR. If they do not, the benefit comes from visual-regime stratification rather than geography, and the method generalizes beyond coordinate-tagged data.

Finally, every headline number is reported under both label-free and oracle model selection. This follows the warning from WILDS 2.0 that adaptation gains often vanish under realistic tuning.