Change Transplants: Teaching Change Detectors to Ignore Seasonal and Lighting Differences

Satellite change detectors compare two images of the same place taken at different times and mark what physically changed, such as a new building or a cleared field. They raise many false alarms because most of the difference between two such images is not structural. Grass browns, shadows swing with the sun, soil dries, haze comes and goes. Remote-sensing researchers call these "pseudo-changes." We argue that this is the same failure that object recognition calls background shortcut learning, where a model predicts from the scenery instead of the object. In both cases the label happens to correlate with the surroundings in the training data, so the model learns the correlation instead of the object.

We propose to break that correlation at the data level with change transplants. A real change instance, meaning the matched before and after pixels of a building footprint, is cut out and pasted into a different location's before/after background pair. That background pair carries its own real seasonal and illumination differences. We also create null transplants: unchanged regions with the same footprint shapes and the same paste seams, labeled "no change." Together the two make paste boundaries and raw background differences uninformative about the label. The same machinery doubles as a diagnostic, a pseudo-change analogue of the background-reliance tests in Noise or Signal: The Role of Image Backgrounds in Object Recognition, which measures how much a change detector's output is driven by its surroundings.

Core Claim

Current pseudo-change methods remove nuisance appearance inside a single image pair. FeaSpect (Feature Spectrum Learning for Remote Sensing Change Detection) swaps Fourier amplitude, the frequency-domain component that carries global "style," between the before and after feature maps of the same tile. FINO (Leveraging Fine-Grained Information and Noise Decoupling for Remote Sensing Change Detection) filters illumination noise and shape noise in separate network branches. Attention-based Siamese models such as ChangeTitans (Towards Remote Sensing Change Detection with Neural Memory) learn to down-weight environmental channels.

None of these changes the statistics of the training set. Three regularities therefore remain available to a detector. True changes tend to sit inside particular landscapes. They tend to coincide with particular kinds of background difference, such as bare soil at construction sites. And the overall magnitude of appearance difference is itself predictive of the label.

Our claim is that recombining real change instances with real, spatially structured pseudo-change backgrounds harvested from the datasets themselves, while matching seams with null transplants, will reduce false alarms under seasonal and cross-dataset shift more than in-pair style alignment does. Spatially structured pseudo-change includes field-level phenology, shadow patterns and moisture patches, which global photometric jitter cannot imitate. We further claim the reduction will be measurable as a smaller pseudo-change gap, without the in-distribution accuracy loss that full context suppression causes.

This differs from the closest existing work in several ways. Unlike FeaSpect, the intervention crosses geographic scenes and acts on spatially local differences rather than global amplitude. Unlike FoBa (FoBa: A Foreground-Background co-Guided Method and New Benchmark for Remote Sensing Semantic Change Detection), which uses unchanged background as a semantic anchor for classifying changed pixels, we treat background as a potential confounder and test whether a model relies on it. Unlike the single-image counterfactual generation of Towards Robust Classification Model by Counterfactual and Invariant Data Generation, our units are temporally paired, so the "background" is itself a before/after pair whose difference is the nuisance.

Research Program

Building the transplant bank. Change detection datasets supply pixel-accurate change masks, such as building polygons in LEVIR-CD and WHU-CD. This largely sidesteps the mask-quality problem that troubles saliency-based background swapping in Distilling Localization for Self-Supervised Representation Learning. We extract two kinds of material.

The first is change instances: connected components of the change mask, each with its before and after pixels. The second is background pairs: crops with no labeled change. We score each background pair by its pseudo-change strength, defined as the mean per-pixel distance between its before and after features under a frozen pretrained encoder. High-scoring pairs are real examples of seasonal, shadow and moisture difference that the annotators judged structurally unchanged.

Let $A_1, A_2$ be the before and after pixels of a source crop, $B_1, B_2$ a background pair from another location, and $M$ a footprint mask. A factual transplant is then

$$\tilde{T}_t = M \odot A_t + (1 - M) \odot B_t, \quad t \in \{1, 2\}, \qquad \tilde{Y} = M,$$

where $\odot$ is pixel-wise multiplication and $\tilde{Y}$ is the change label. A null transplant uses the same equation and the same blending. The differences are that $A_1, A_2$ come from an unchanged region, $M$ is the footprint shape of a real change instance drawn from the bank, and $\tilde{Y} = 0$. Because both kinds of transplant carry identical seams with identical shape statistics, a seam cannot become a shortcut. This addresses the boundary-artifact concern raised against copy-paste methods.

We sample background pairs preferentially from the high pseudo-change end, and from the same dataset, to keep scenes roughly plausible. This responds to the semantic-incoherence criticism of arbitrary pasting.

Rare changes. Following the finding in How Re-sampling Helps for Long-Tail Learning? that oversampling rare examples memorizes their backgrounds, rare change instances are transplanted rather than duplicated. Rarity here can mean small footprints or uncommon structure types. This lets oversampling and context shift happen together, in the spirit of The Majority Can Help the Minority: Context-rich Minority Oversampling for Long-tailed Classification.

Training. Transplants are mixed with original pairs at a ratio $\rho$, which we sweep. We do not remove real context. The tension FoBa raises is real: context legitimately helps. We therefore track in-distribution accuracy as a first-class outcome. The method is a data intervention and can wrap any detector. We apply it to a convolutional Siamese baseline, to a strong attention model, and to the foundation-model adapter BAN (A New Learning Paradigm for Foundation Model-based Remote Sensing Change Detection) using its released code (likyoo/BAN).

Diagnostics. We define three measurements. The first is context-only predictability, the analogue of Xiao et al.'s finding that background-only classifiers reach 40–50% on nine classes. We train a detector on pairs whose change regions have been replaced by inpainted static texture, then measure how well it still localizes the hidden change regions from surroundings alone. The second is the pseudo-change gap: the IoU drop when the same change instances are re-evaluated on held-out high pseudo-change background pairs instead of their original backgrounds. The third is the background false-alarm rate: false-positive rate on held-out high pseudo-change background pairs that contain no change at all.

Extension. If binary results hold, we test semantic "from-to" change detection on LevirSCD. The question there is whether transplants reduce reliance of transition labels on surrounding land use, without undoing the gains FoBa attributes to context.

Baselines

The naive reference is each detector trained with its standard recipe on unmodified data. This gives the in-distribution numbers and the pseudo-change gap we hope to shrink. For scale, ChangeTitans reports 84.36% IoU and 91.52% F1 on LEVIR-CD (Towards Remote Sensing Change Detection with Neural Memory). We will retrain it in our pipeline rather than rely on the reported figure. Matching or slightly exceeding its in-distribution IoU while cutting the pseudo-change gap would count as success. A gain on the gap paired with an in-distribution loss greater than about one IoU point would count as a trade-off, not a win.

The strongest pseudo-change competitor is FeaSpect, which aligns style inside each pair. It reports a 3.94% IoU improvement over ConvNet baselines on SVCD and a 9.88% IoU improvement when added to STANet. The file gives these only as relative gains; no absolute SVCD IoU is reported in it. We will run FeaSpect and transplants on the same backbone, data splits, epochs and hyperparameter search budget. We will also run them combined, because the two are not mutually exclusive. Beating FeaSpect means a lower background false-alarm rate and higher cross-dataset IoU on SVCD and in transfer. A combination that beats both components would show the two interventions address different confounds.

FINO is the strongest decoupling-by-architecture competitor. It reports +4.97% IoU on WHU-CD and +1.03% IoU on LEVIR-CD. We compare against it on those two datasets under matched budgets.

For out-of-domain robustness the reference is BAN. Its BAN-BIT variant reports an 11.96% IoU improvement over plain BiT on the BANDON out-domain test set. No absolute BAN IoU on LEVIR-CD was found in the file. Beating it means BAN plus transplants exceeding BAN alone on BANDON out-domain at identical adapter capacity and training length.

Two ablations isolate the key ingredients. Transplants without null transplants test whether seam-matched negatives matter; we expect seams to become a shortcut and false alarms to rise along paste boundaries. Transplants using random background pairs rather than high pseudo-change ones test whether real structured pseudo-change is the active ingredient.

Why not just…?

Why not just apply strong, independent color and brightness jitter to the before and after images? This is the most serious threat. If pseudo-change were mostly global illumination, jitter would cover it at almost no cost. The case against is that seasonal and shadow differences are spatially structured: one field greens while its neighbour does not, and shadows fall in building-shaped patterns. Global jitter cannot synthesize these. FeaSpect's authors also report that hand-tuned pixel-level transformations generalized poorly compared with their learned alignment. We do not take that as settled. We will run a tuned per-time jitter baseline with the same search budget as transplants. If jitter closes most of the pseudo-change gap, the structured-pseudo-change hypothesis is wrong and we will say so.

Why not just translate the before image into the after image's season with an image-to-image generator? Generator-based normalization, as in CDGAN-style approaches, is costly. FeaSpect's authors note it introduces synthetic artifacts that detectors read as change. It also acts only within the pair, so like FeaSpect it leaves dataset-level correlations between landscape and change untouched. Transplants use only real pixels. Given these documented problems we do not run a generator baseline. FeaSpect stands in as the stronger in-pair alignment method.

Why not just rely on a frozen foundation model, which already sees enormous background variety? Compositional by Design: Background-Invariant Representations via Linear Additivity in VLMs reports that CLIP-family models still rely on background shortcuts. BAN needed task-specific adapters to transfer well. Our experiment measures this directly: we compute the pseudo-change gap for BAN with and without transplants. If BAN's gap is already near zero, transplants are unnecessary for foundation-model detectors and the contribution narrows to smaller models.

Why not just oversample rare changes or reweight the loss? Shi et al. show plain re-sampling falling below ordinary cross-entropy on CIFAR100-LT (34.1% versus 38.3%), while context-shifted resampling reaches 45.8%. The reason is that duplicating rare examples duplicates their backgrounds. We include class-balanced resampling and focal loss as baselines for small and rare changes, so the same effect can be checked for change detection.

Proposed Experiments

In-domain and seasonal. We train and test on LEVIR-CD, WHU-CD and SVCD, reporting IoU and F1 on each dataset's own test split.

Cross-dataset shift. We train on LEVIR-CD and test zero-shot on WHU-CD, S2Looking and SVCD. We also evaluate on the BANDON out-domain split through the BAN codebase.

Diagnostics. For every trained model we report context-only predictability, the pseudo-change gap and the background false-alarm rate. All three are computed on held-out background pairs never used to make transplants.

Decisive checks. The first check comes before any method work. If a context-only detector localizes changes at chance level, there is no shortcut to fix, and the project reduces to a negative diagnostic result.

The second check concerns where the benefit comes from. If transplants beat tuned per-time jitter and FeaSpect on background false alarms and cross-dataset IoU, the claim holds. If removing null transplants erases the benefit, or if random background pairs work as well as high pseudo-change ones, then the benefit lies elsewhere than we claim.

The third check is the trade-off curve over the mixing ratio $\rho$. It tells us whether pseudo-change robustness can be bought without paying in in-distribution accuracy, which is the trade-off this proposal is judged on.