Holding State Through the Clouds: Change Detection When Half the Picture Is Missing

Flood and disaster mapping compare a "before" and an "after" satellite image to find what really changed, such as new water or collapsed buildings, while ignoring shadows, seasons, and sensor quirks. Two problems usually arrive together in practice. The two images often come from different sensors, typically an optical camera and a radar (Synthetic Aperture Radar, SAR, which sees through clouds but looks nothing like a photograph). The optical image is also often partly hidden by cloud or haze, sometimes over most of the scene. Current methods handle only one of these problems at a time. Cross-sensor change detectors assume a fully visible scene. Cloud-removal methods restore a single image and never decide what changed.

We propose a change detector built on a state space model. This is a sequence model that scans an image as a long chain of patches and carries a running memory along the chain at cost linear in image size. The detector gets one structural change: at every patch it predicts how trustworthy the observation is, and that trust directly controls how much the patch may overwrite the memory. On cloud-covered patches the model writes nothing and carries forward evidence from visible regions, which can be far away. It also tracks how long it has been carrying memory without fresh evidence, and abstains when that gap grows too large. The bet is that, under heavy occlusion, refusing to write corrupted evidence into a global memory beats both inventing the missing pixels and simply masking them out.

Core Claim

A state space model updates its memory $h_t$ at patch $t$ through a step size $\Delta_t$ that it computes from the input. With continuous dynamics $A$ and input map $B$, the discretized update is $h_t = \bar{A}_t h_{t-1} + \bar{B}_t x_t$, where $\bar{A}_t = \exp(\Delta_t A)$. We multiply the step size by a predicted reliability $r_t \in [0,1]$:

$$\bar{A}_t = \exp(r_t \Delta_t A), \qquad \bar{B}_t \approx r_t \Delta_t B .$$

When $r_t = 0$, $\bar{A}_t$ becomes the identity and $\bar{B}_t$ becomes zero, so the memory passes through the occluded patch unchanged and nothing from the cloud is written. A second quantity is tracked along each scan: the number of patches since the last reliable write, $a_t$. It costs nothing to compute and gives an uncertainty signal for abstention that comes from the structure of the scan, not from a separate calibration head.

The claim is specific and testable. On heterogeneous optical–SAR change detection with realistic occlusion above roughly 40% of the optical image, three things should hold. First, reliability-gated scanning should give higher change F1 on visible pixels than any of the alternatives. Second, it should give higher F1 on occluded pixels at matched coverage. Third, its error under abstention should be better calibrated than restoring the image before detection, feeding the occlusion mask as an extra input channel, or ignoring the optical image altogether.

Relation to Prior Work

The closest single paper is DehazeMamba. It uses a Mamba backbone and a haze-weight map that sends SAR features only into hazy zones, and it restores cloudy Sentinel-2 images to 33.39 dB PSNR on MRSHaze. It is single-date restoration: it never compares two dates, and its reliability map routes features rather than controlling what the scan remembers. We borrow its idea of an occlusion-aware weight map and move it inside the recurrence of a change detector.

The cross-sensor side is set by M²CD. It shows that a shared backbone fails across optical and SAR, and it fixes this with modality-specific experts and an optical-to-SAR self-distillation branch, using quadratic-cost Transformer or CNN encoders on fully visible scenes. The linear-cost side is set by CDMamba and ChangeRWKV. These show that global linear-time models give clean change maps at a fraction of Transformer cost, but only on same-sensor, fully observed pairs. CD-Lamba shares that limitation.

The obvious combination, an M²CD with a Mamba backbone, is therefore not what we propose. It would inherit M²CD's assumption that the scene is fully visible, and swapping a backbone is not a research question. Our contribution is the gating of memory writes plus abstention driven by memory age. Neither exists in any of these papers.

Two findings from inpainting constrain the design. ZITS++ shows that above about 50% masking, purely texture-based filling collapses structurally (PSNR 26.15 versus LaMa's 25.20 on Indoor). Erase Diffusion shows that generative fillers with scarce context tend to regenerate what was there before. For change detection, that bias toward old content is especially dangerous: an inpainter that fills the occluded area with the pre-event appearance will hide exactly the changes we want to find. This is why we gate rather than fill.

Medical longitudinal work points the same way. The BraTS-Reg challenge reports that registration error spikes within 30 mm of resection cavities, because some tissue has no counterpart at the other time point. We treat that as the medical analogue of cloud cover and leave it to a follow-up, since none of the benchmarks in hand provides longitudinal labels for it.

Research Program

The model has three parts.

The first part is two modality-specific stems, one for optical and one for SAR, as light versions of M²CD's experts. They map each image into a shared patch embedding.

The second part is a reliability head that predicts $r_t$ for every patch of each image. It is trained with a weak supervisory signal: synthetic occlusion masks during training, plus a cross-modal consistency term that lowers $r_t$ wherever the optical embedding disagrees with what the SAR embedding predicts for that location. On SAR the same head can learn to down-weight speckle-dominated patches, so the mechanism does not depend on which modality is occluded.

The third part is a four-direction scanning state space encoder in the style of CountMamba and RS-Mamba, with the gated discretization above and a convolutional side branch for local detail as in CDMamba's design. Scanning in several directions matters: memory held across a cloud then reaches each occluded patch from several visible borders, not just one.

Temporal fusion interleaves the two dates' patches within the scan, so that comparing the two dates happens inside the recurrence. A separate cross-attention module would bring back quadratic cost. The decoder outputs, per pixel, a change probability and an abstention score computed from the change logit together with the four directional memory ages $a_t$.

The analysis asks three questions. Does the learned $r_t$ track real haze it was never trained on? How does accuracy on occluded pixels decay with distance to the nearest visible pixel? Do floods, which are spatially contiguous, benefit more from held memory than building changes, which are scattered? That last question checks whether the gain really comes from long-range context or merely from avoiding corruption.

Baselines

The naive reference point is the strongest same-sensor linear model, CDMamba, run unchanged on occluded inputs. It reports 93.76% F1 and 88.26% IoU on WHU-CD with clean inputs. It shows how much occlusion costs a good model that has no idea the input is corrupted.

The strongest cross-sensor competitor is M²CD-MiT-b1. It reports 91.96% mF1, 85.66% mIoU and 96.19% OA on unoccluded CAU-Flood. We will first reproduce it on clean CAU-Flood within one point of these figures, then retrain it on the occluded variant with the same occlusion sampling, epochs, and hyperparameter search budget as our model. No result exists for M²CD on occluded CAU-Flood; Lacuna lists nothing for that setting, so we produce it ourselves.

The restore-then-detect pipeline uses DehazeMamba-style SAR-guided restoration of the optical image followed by M²CD. We will train the restorer on the same occlusion masks. Its 33.39 dB on MRSHaze shows it is a serious restorer, not a strawman.

The SAR-only reference drops the optical image wherever it is occluded and detects change from the SAR evidence alone.

The ablation that isolates our key ingredient is the identical architecture with the gate removed and the reliability map concatenated as an extra input channel. That baseline can in principle learn to shrink $\Delta_t$ on its own. A second ablation keeps the gate but removes the memory-age term, to test whether abstention needs it.

All models get matched parameter counts within 20% and the same training data and augmentations. Each gets the same number of hyperparameter trials, and results are reported over three seeds. Beating the field means two things on the occluded CAU-Flood test split at 40–80% coverage. Our model must exceed the best baseline's mF1 on occluded pixels by more than the spread across seeds. It must also give a lower area under the risk–coverage curve, without losing more than half a point of mF1 on the clean split.

Why not just…?

Why not just restore the optical image with SAR guidance, then detect changes? This is the most serious threat, because DehazeMamba shows SAR-guided restoration is strong. Our concern is specific. A restorer is trained to produce plausible images, not to preserve evidence of change, and the regeneration bias reported by Erase Diffusion pushes fills toward pre-event appearance. The predicted signature is that restore-then-detect loses most of its recall on occluded pixels that truly changed, while precision stays roughly intact. We will report recall separately for changed and unchanged occluded pixels. If the pipeline matches us there, the gating idea is unnecessary, and we will say so.

Why not just use the radar image wherever the optical one is cloudy? If SAR alone carries enough information about change, gating optical evidence adds nothing. M²CD's results show that optical–SAR fusion beats naive shared processing, but they do not show that optical evidence matters once SAR is present. The coverage sweep settles this question directly. If SAR-only matches our model on visible pixels at every coverage level, the cross-sensor half of the claim fails, and the proposal reduces to the single-sensor occlusion study on LEVIR-CD and WHU-CD.

Why not just give a standard Mamba change detector the cloud mask as an input channel? Mamba's step size already depends on the input, so a model given the mask could learn to stop writing on cloudy patches by itself. This is exactly the ablation above. Hard gating is expected to win under coverage the model never saw in training and under real haze transplanted from MRSHaze. A learned soft gate has no guarantee of holding memory when the input looks unfamiliar, while multiplying the step size by zero holds it by construction. If the extra-channel variant matches us on held-out coverage and real haze, the contribution shrinks to the abstention signal alone.

Why not just mask occluded pixels out of the loss? Masking prevents corrupted supervision but still lets corrupted features flow through the model's memory to visible pixels. It also produces nothing at all for occluded pixels. The BraTS-Reg evidence that errors cluster around voids suggests the damage spreads beyond the masked region. We will measure this by comparing F1 in a 32-pixel ring around each occlusion against F1 far from it.

Proposed Experiments

The main benchmark is CAU-Flood (15,231 / 3,807 / 3,071 pairs) with occlusion transplanted onto the optical image at coverage from 0% to 80% in steps of 10%. We use two kinds of occlusion. Synthetic blob masks are used for training. Real haze patterns extracted from MRSHaze's Sentinel-2 images are used only for testing. Metrics are mF1, mIoU and OA, reported separately for visible pixels, occluded pixels, and the ring around each occlusion. We also report the area under the risk–coverage curve and expected calibration error for abstention.

A same-sensor check on LEVIR-CD and WHU-CD, with the same synthetic occlusion, tests the gate without SAR. The reference points are ChangeRWKV-B (92.16% F1 on LEVIR-CD) and CDMamba (93.76% F1 on WHU-CD). We also report FLOPs and memory at 1024×1024 input, against ChangeRWKV-B's 134.25G FLOPs and 6.5 GB.

Three outcomes are decisive. First, if the gated model does not beat the extra-channel ablation on real-haze test occlusion, the core mechanism is not needed. Second, if restore-then-detect matches our recall on changed occluded pixels, filling the missing region is as good as gating it. Third, if accuracy on occluded pixels does not fall more slowly with distance to visible evidence for floods than for buildings, then long-range memory is not what drives the gain. Any one of these would falsify a named part of the claim. We will report all three regardless of outcome.