Training-Only Geography for Dense Semi-Supervised Segmentation

Satellite and aerial images almost always come with metadata: roughly where and when they were taken. That metadata is a strong hint about which land-cover classes to expect. A tile from a rural district is more likely to contain farmland than parking lots. Expert pixel labels for these images are scarce and expensive, so the hint is tempting to use. It is also noisy, in ways that break simple use. Coordinates can jump, district tags can be wrong or too coarse, and a model that leans on location can fail badly when deployed somewhere new. This note proposes using geography only when generating training labels, in a form that survives noisy metadata. The finished segmentation model never sees metadata and must instead learn to infer the same context from the whole image.

The proposal has two parts. The first is a teacher network that sees image plus location and produces pixel pseudo-labels. Before a label is trusted, its output is averaged over a realistic, heavy-tailed model of metadata error, so no label depends on one possibly wrong coordinate. The second is an image-only student network with a full-image receptive field, built from a linear-time recurrent architecture so it can process 1024×1024 tiles at native resolution. The bet is that the student can only absorb the teacher's location-informed decisions if it sees enough of the tile to recover the missing context from visual evidence such as settlement density or field patterns. Small-window students should not be able to. Both parts are testable on LoveDA with controlled metadata corruption.

Core Claim

Geographic metadata improves dense semi-supervised segmentation only when two conditions hold. First, pseudo-labels are made robust to the actual error distribution of the metadata, including discontinuous jumps and not only small jitter. Second, the image-only student has a receptive field large enough to infer what the metadata told the teacher. If either condition is removed, the metadata gain should vanish or turn negative. Because the student needs global context at high resolution, sub-quadratic global models are a requirement here rather than an efficiency nicety. Windowed or purely convolutional students will waste the signal.

What exists and what is missing

The closest work is Context Matters. It feeds geolocation and acquisition day into a Vision Transformer (ViT) teacher as an extra "metatoken," trains an image-only student on the teacher's pseudo-labels, and reaches 46.65% mAP on BigEarthNet with 1% labels against 43.09% for the best conventional semi-supervised baseline. We reuse its teacher–student split and its central warning: a student that is directly conditioned on metadata overfits and breaks under spatial shift. We depart from it on three points. It assumes clean Sentinel-2 header coordinates, while we model metadata error explicitly. It does scene-level multi-label classification, while we do pixel segmentation. Its student is a quadratic-cost ViT, while ours is a linear-time global model whose receptive field is part of the hypothesis.

GPS-GLASS motivates the noise model. It transfers daytime segmentation labels to night scenes by matching frames through GPS, then throws away pixels where GPS or visual matching disagree through a heuristic confidence map. Discarding is exactly the coverage-versus-noise trade-off that the research file flags as unresolved. We replace hard filtering with averaging over a stated error distribution.

Presence-Only Geographical Priors multiplies an image classifier's output by a learned location prior, lifting iNaturalist top-1 accuracy from 50.17% to 58.44%. It is the natural cheap competitor, and we run it as a baseline applied to pseudo-labels. Learning to Segment Medical Images with Scribble-Supervision Alone supplies the method template: it uses Monte Carlo dropout uncertainty to stop self-training from reinforcing its own mistakes. We apply the same logic, but the random variable is the metadata error rather than the network weights.

On the architecture side, Beyond Quadratic: Linear-Time Change Detection with RWKV shows that a recurrent model with linear cost in the number of tokens handles 1024×1024 remote-sensing inputs at 134.25G FLOPs and 6.5 GB peak memory, against 852.44G FLOPs and 10.2 GB for a Transformer. This model is called RWKV, short for Receptance Weighted Key Value, and it replaces attention with a decaying running summary. That work is fully supervised, and its authors point to weak supervision as future work. We supply that setting. SG-Former argues that windowed attention gives up long-range context. LookupViT notes that heavy token compression has not been shown to work for dense prediction. Together these constrain the student design: it needs a global receptive field without pooling away the high-resolution grid. The known limitation of scan-order bias in recurrent vision models is a real risk, and we address it below with multi-direction scans.

Research Program

The teacher is a hybrid segmentation network that takes the image together with a metadata embedding. Following Context Matters, the metadata enters early as an extra token. Its location input is encoded with sine and cosine features as in the presence-only priors work. On LoveDA, the available metadata is the administrative district (18 districts) and the urban or rural split. Approximate coordinates come from district centroids, so the metadata is coarse and noisy by construction.

During teacher training, the metadata is corrupted with an error model $q$. With probability $1-\epsilon$ it receives small Gaussian jitter. With probability $\epsilon$ it is replaced by a uniformly random location or district, which mimics the discontinuous multipath jumps and mis-tags that Gaussian filtering cannot represent. At pseudo-labeling time, the teacher's prediction for pixel $u$ is averaged over $K$ draws from the same error model:

$$\bar p_u(y) = \frac{1}{K}\sum_{k=1}^{K} p_\theta(y \mid I, \tilde x + \delta_k)_u, \qquad \delta_k \sim q.$$

Here $I$ is the image, $\tilde x$ is the observed metadata, $\delta_k$ is the $k$-th sampled error, and $p_\theta$ is the teacher's per-pixel class distribution. A pixel's pseudo-label is the argmax of $\bar p_u$. It is kept only if the maximum exceeds a confidence threshold $\tau$ and the argmax agrees across at least a fraction $\rho$ of the draws. In words, a pixel's label must not depend on the metadata being exactly right.

The error rate $\epsilon$ is fitted on the labeled split. We choose it to maximize the likelihood of the labeled pixels under the averaged teacher, which lets the data decide how far to trust the metadata.

The student is image-only. Its encoder is hierarchical in the style of RWKV, scanning in four directions to reduce bias along the cardinal axes. Its decoder is lightweight, with high-resolution skip connections so thin structures such as roads are not smoothed away. The student trains on labeled pixels plus the kept pseudo-labels, with the usual weak-to-strong augmentation consistency used in semi-supervised segmentation. The teacher is refreshed from the student's features each round, while metadata remains a teacher-only input.

Baselines

The naive reference point is supervised training of the same student on the labeled fraction alone. Its job is to show how much any unlabeled data helps.

The strongest metadata-free competitor is standard confidence-thresholded self-training of the same kind S5 uses, with an identical student, augmentations, and number of unlabeled passes. If metadata adds nothing to dense segmentation, this baseline will match us.

For scale, S5 reports 55.65 mIoU on LoveDA with a 600M-parameter ViT-H after large-scale semi-supervised pre-training. LoveDA itself reports 49.79% test mIoU for fully supervised HRNet and 41.32% for CBST self-training in rural-to-urban adaptation. We will not match S5's compute, so its number is an upper reference rather than a head-to-head target. I found no reported result on Lacuna for semi-supervised LoveDA segmentation at 1–10% label fractions, so all low-label comparisons will be run by us under one shared protocol.

The strongest metadata-using competitor is Context Matters extended to segmentation. It uses the same teacher with a metatoken but no error model and no averaging, and thresholds its pseudo-labels directly. We also run a late-fusion variant: an image-only teacher whose pixel outputs are multiplied by a separately trained district-level class prior, as in the presence-only priors work.

Two ablations isolate the proposal's ingredients. The first keeps everything fixed but uses Gaussian-only corruption, testing whether the heavy-tailed part of the error model matters. The second keeps the pseudo-labels fixed and swaps the student among the RWKV-style global model, a Swin-style windowed Transformer, and HRNet, testing the receptive-field claim.

All methods share the same splits, unlabeled pool, number of training iterations, and a fixed hyperparameter budget of twenty trials tuned on a held-out labeled validation district. Beating the field means two things: a higher mean IoU on held-out districts than the best of these baselines at 1% and 5% labels across three seeds, and staying at or above the metadata-free baseline at every corruption level.

Why not just…?

Why not just ignore the metadata and run strong self-training? This is the most serious threat. LoveDA's urban and rural scenes may be visually obvious, which would make district tags redundant. S5 shows that metadata-free semi-supervised pre-training already beats self-supervised foundation models on LoveDA. The metadata-free baseline above decides this directly. If our gain over it is within seed noise on held-out districts, we will report that metadata is redundant for this task and fall back to BigEarthNet, where Context Matters found a 3.6-point gain at the scene level.

Why not just reuse Context Matters with a segmentation head? Its teacher was trained on clean coordinates, so on corrupted metadata it will confidently mislabel whole tiles. Whole-tile errors are exactly the confirmation-bias failure that self-training amplifies. The corruption sweep below tests this. Our prediction is that it matches us at $\epsilon=0$ and falls below the metadata-free baseline as $\epsilon$ grows, while our method converges toward that baseline.

Why not just multiply pixel predictions by a location prior? Late fusion assumes appearance and location are independent given the class. Context Matters reports that early fusion beats late fusion on its benchmarks. That finding comes from classification, though, and late fusion is cheap enough that we run it as a full baseline rather than assume it loses.

Why not just add Gaussian noise during teacher training? This handles jitter but not discontinuous jumps or wrong district tags, which are the failure modes the research file identifies. The Gaussian-only ablation, evaluated under jump-type corruption, settles the question.

Why not just use a CNN or windowed-attention student? These are cheaper and well understood. Our claim, however, is that the student must infer tile-wide context to absorb the metadata-informed labels. The student swap tests exactly this. If HRNet gains as much from the metadata pseudo-labels as the global student does, the receptive-field half of the claim is wrong, and we will say so.

Decisive Checks

The main experiment trains on LoveDA with 1%, 5%, and 10% of labeled tiles and evaluates mean IoU on districts held out entirely from training. Holding out districts is the spatial-shift regime in which Context Matters warns that metadata conditioning fails.

A corruption sweep injects mixed Gaussian and jump noise into district and coordinate metadata at $\epsilon \in \{0, 0.1, 0.3, 0.5\}$ and plots each metadata method's gain over the metadata-free baseline. The proposal predicts a gain that stays positive and shrinks gracefully toward zero, while the naive metatoken teacher goes negative.

The receptive-field test reports the metadata gain separately for each student architecture under identical pseudo-labels. It adds a linear probe that predicts district and urban or rural from frozen student features, testing whether global students actually recover the metadata from image context. The probe is the mechanism check: high probe accuracy together with a large gain supports the story, while a gain with no probe signal would point to a different explanation.

Cost is reported as FLOPs and peak memory at 1024×1024 against the windowed student, measured in the same way as ChangeRWKV.

A smaller bridge experiment on BigEarthNet, which has true coordinates and dates, reproduces the 1% setting of Context Matters. It checks that the averaging step does not hurt when metadata is clean, and that it helps once we corrupt it.