Satellite change-detection models can usually tell that something changed. They are much worse at saying which way it changed: forest to bare soil or bare soil to forest, flooding or draining, construction or clearing. The problem gets worse in regions the model has never seen. On the global daily satellite benchmark DynamicEarthNet, the best spatio-temporal models score only about 10–11% on detecting where and when change happens. Counterintuitively, they do worse when given daily image sequences than when given weekly ones (DynamicEarthNet). This is a temporal ambiguity problem of the same kind that video researchers call "chiral" actions, meaning pairs like opening versus closing that pass through the same intermediate states in opposite order. It also sits inside a geographic shift problem, because the same land-use class looks spectrally different from continent to continent.
We propose treating time reversal as a data symmetry that changes the label, in the same spirit as horizontal mirroring combined with a mirrored robot action. Played backwards, a sequence of forest turning into soil is a plausible sequence of soil turning into forest. The change mask stays the same, while the "from" and "to" labels swap. Combined with the eight rotations and reflections of the square image grid, this doubles the scarce labeled change events. It also forces the model to encode change direction as something that flips sign when time runs backwards, which static, region-specific appearance cues cannot do. The catch is that time reversal is only an approximate symmetry. Vegetation green-up reversed looks like autumn senescence, and gradual construction reversed looks unlike real, abrupt demolition. The central technical contribution is therefore to apply the reversal constraint only where it is physically admissible, and to separate the region's own seasonal drift from the directional change signal before applying it.
Core Claim
A change representation trained to be equivariant under the joint group of grid rotations/reflections and time reversal will predict change direction more accurately on held-out continents than the same architecture trained with ordinary augmentation. Equivariant here means that rotating the input rotates the output, and reversing time negates the change direction and swaps the from/to classes. The equivariance has to be gated by a learned test of whether a reversed sequence is distinguishable from a real one, and applied to a direction signal from which the scene's shared seasonal trend has been removed. Each of the three ingredients (reversal, admissibility gating, seasonal removal) should be separately necessary. If plain reversed-clip augmentation matches the full method, the claim is false and we will report it as such.
How the idea relates to prior work
Two papers supply the method. Chirality in Action introduced LiFT, which compresses a video into a static token $z_s$ and a direction token $z_d$ under a straight-line latent trajectory. LiFT reaches 86.6% on SSv2-Chiral against 80.4% for VideoJEPA and 80.3% for VideoMAE. It works on pooled whole-frame features from indoor video. It does not use time reversal as a training symmetry. It also assumes the background is static, which seasonal phenology violates. MirrorDuo showed that a spatial reflection is only useful when the coupled quantity (the robot action) is transformed with it. Doing so lifted mirrored-domain success from 3.3% to over 70%. MirrorDuo also found that visual asymmetries break the symmetry and must be handled. We carry this coupled-transformation logic from the spatial axis to the time axis, where the coupled quantity is the from/to label pair. We also turn MirrorDuo's asymmetry warning into an explicit admissibility gate. TrajTok similarly mirrors driving trajectories to remove left/right turn bias, but it works on coordinates, not imagery.
Three papers motivate the proposal. DynamicEarthNet documents the weak change-detection scores and the daily-versus-weekly paradox. LiFT's authors report that image-backbone features separate vertical motion far better than horizontal motion. That anisotropy should make overhead change detection depend on landscape orientation, which the grid-symmetry orbit is meant to remove. Position: Satellite Data is a Distinct Modality argues that overhead imagery has no canonical "up", that cross-region accuracy can drop by up to 30%, and that satellite-specific pretraining beats ImageNet pretraining by up to 18%. On that basis we use a satellite-pretrained frame encoder where available.
Two papers constrain the design. WILDS found that IRM and CORAL failed to beat plain training on 8 of 10 real shifts, which steers us away from generic invariance penalties. Trusting Small Training Dataset for Supervised Change Detection found that geographic diversity beats proximity on OSCD, with Cohen's Kappa of 0.32 versus 0.12. We therefore build geographically diverse training splits rather than assuming local data is best. Koopman-Assisted Trajectory Synthesis warns that augmentations which ignore the system's dynamics produce implausible trajectories. Gating reversal by admissibility is our answer to that warning.
Research Program
The model encodes each daily frame with a frozen image encoder and keeps patch tokens rather than one pooled token, since change in satellite scenes is local. A small temporal Transformer then produces, for every patch, a static token $z_s$ and a direction token $z_d$ under the LiFT parameterization $z_t = z_s + (t/T)\,z_d$, where $t$ indexes the frame and $T$ is the window length. Fitting a straight line through roughly 30 daily frames acts like a robust regression that should average out cloud and illumination noise. That is our proposed explanation of the daily paradox, and it is directly testable.
To keep seasons out of the direction token, we estimate a scene-level seasonal direction $\bar z_d$ as a robust median of $z_d$ over all patches in the area of interest. This works because only about 5% of pixels change in a given month, so the median is dominated by unchanged land. The direction signal used for prediction is the residual $r = z_d - \bar z_d$. Three heads predict the change mask from the size of $r$, the "from" class from $z_s$, and the "to" class from $z_s + z_d$.
Under time reversal $\tau$, the straight-line parameterization implies an exact expected transformation of the latents: the reversed sequence should start where the forward one ended, with the opposite direction. The reversal consistency loss is
$$ \mathcal{L}_{\tau} = w(x)\,\Big( \lVert z_s(\tau x) - z_s(x) - z_d(x) \rVert^2 + \lVert r(\tau x) + r(x) \rVert^2 \Big), $$
where the change mask of $\tau x$ equals that of $x$ and its from/to labels are swapped. The weight $w(x)$ comes from a separate small "arrow-of-time" classifier trained to tell forward patch sequences from reversed ones. We set $w = 1 - |2p - 1|$, where $p$ is that classifier's probability of "forward". Reversal is therefore enforced where the classifier cannot tell the direction and switched off where it can, because in those cases the reversed sample is out of distribution and would teach a shortcut. For each of the eight grid symmetries $g$, we apply a parallel consistency loss requiring that the outputs on $g x$ equal $g$ applied to the outputs on $x$, with masks transformed accordingly and labels unchanged.
Beyond accuracy, the program produces two analyses. The first is a map of which land-cover transitions the arrow-of-time classifier judges reversible, broken down by continent. This gives a direct empirical answer to how symmetric land change actually is. The second is an orientation-sensitivity profile: the spread of direction accuracy across the eight rotated and reflected copies of each test sequence. It tests whether the horizontal-motion gap reported for LiFT appears in overhead imagery and whether the grid orbit removes it.
Baselines
The naive reference point is the spatio-temporal segmentation model U-TAE from the DynamicEarthNet study, trained with the authors' weekly subsampling and ordinary random flips, with no reversal and no seasonal residual. On the official split, the DynamicEarthNet paper reports Binary Change around 10–11%, Semantic Change 25–28%, and 37.9% test mIoU for purely supervised monthly training. We rerun it on our geographic splits, because those numbers are not continent-held-out.
The strongest existing method on this benchmark is the semi-supervised Context-Aware Consistency model, with 43.6% test mIoU on daily data according to DynamicEarthNet. It is the strongest competitor because it already exploits unlabeled daily frames, which is a large part of what our model does. We will rerun it on identical splits with the same tuning budget. Beating it means higher Semantic Change and Binary Change on held-out continents across three seeds, with non-overlapping confidence intervals. Matching mIoU alone does not count, because mIoU is dominated by unchanged pixels.
A per-patch LiFT with the same encoder and temporal Transformer, but no symmetry losses, isolates the contribution of the symmetries from the straight-line trajectory model itself.
The most important baseline is plain time-reversal augmentation. It adds every reversed sequence with swapped labels to training, with no gating and no seasonal residual. It is the cheapest version of our idea, and the full method must beat it to justify itself.
The remaining ablations each remove one ingredient: the gate (set $w=1$), the seasonal residual (use $z_d$ directly), the grid orbit, and the reversal term.
All variants share the frozen encoder, the training schedule, an equal hyperparameter-search budget of the same number of trials per method, and the same GPU hours.
Why not just…?
Why not just add reversed sequences with swapped labels as extra training data? This is the real threat, and the proposal is designed so it can win. MirrorDuo found that symmetry augmentation helps when the symmetry holds and introduces artifacts when it is broken by visual asymmetries. Land change is asymmetric in exactly this way: growth is gradual and clearing is abrupt. Our prediction is that ungated augmentation helps on transitions the arrow-of-time classifier finds reversible and hurts on the rest, mostly phenology-driven and abrupt-event transitions. Breaking held-out direction accuracy down by transition type will settle it. If ungated augmentation is equal everywhere, we drop the gate and report the simpler method.
Why not just subtract the scene's average trajectory, i.e., classical detrending? Our seasonal residual is a detrending step, so this is less an alternative than an ingredient to isolate. Detrending applied to the inputs of the U-TAE baseline will be tested directly. If input-space detrending plus plain reversal augmentation matches the full model, the learned latent machinery is unnecessary, and the contribution reduces to the augmentation finding.
Why not just use a domain-generalization loss such as IRM or CORAL with continents as domains? WILDS found these failed to beat plain training on 8 of 10 real-world shifts, including geographic ones. We still run CORAL with continent labels as a cheap check under the same budget. However, we expect it to address appearance shift rather than direction ambiguity.
Why not just use a large pretrained video model? On SSv2-Chiral, VideoMAE and VideoJEPA trail the 8.7M-parameter LiFT by about six points (Chirality in Action). RESOUND shows that large datasets like Kinetics carry strong static bias. No reported result on Lacuna was found for large video models on DynamicEarthNet. We therefore include a frozen VideoMAE-style encoder with the same heads as a reference, without assuming the outcome.
Proposed Experiments
The primary benchmark is DynamicEarthNet, re-split leave-one-continent-out across its 75 areas of interest. Training continents are mixed for diversity, following the OSCD finding. We report Binary Change, Semantic Change, and mIoU. We also report a new direction accuracy metric: on truly changed pixels, the rate of predicting the from/to order correctly, which chance would put at 50% for a given transition pair. As a secondary check we use OSCD, where with two frames reversal simply swaps the image pair, to test the bi-temporal case, and SSv2-Chiral, to test that the mechanism does not hurt on natural video. There we compare against LiFT's reported 86.6%, using the global-token variant of our losses.
Three decisive checks determine whether the claim stands. First, on held-out continents, the full model must beat both ungated reversal augmentation and Context-Aware Consistency on Semantic Change and direction accuracy. Second, with our model, daily input must equal or exceed weekly input; if the paradox persists, the straight-line-fit explanation is wrong. Third, the spread of direction accuracy across the eight grid transforms of each test sequence must shrink substantially relative to unconstrained LiFT, showing that orientation bias exists in overhead data and that the orbit removes it. If the first check fails while the third succeeds, the honest conclusion is that rotation symmetry fixes orientation bias but time reversal does not help direction. That would still be a useful negative result.
References
- DynamicEarthNet: Daily Multi-Spectral Satellite Dataset for Semantic Change Segmentation
- Chirality in Action: Time-Aware Video Representation Learning by Latent Straightening
- MirrorDuo: Reflection-Consistent Visuomotor Learning from Mirrored Demonstration Pairs
- TrajTok: What makes for a good trajectory tokenizer in behavior generation?
- Position: Mission Critical – Satellite Data is a Distinct Modality in Machine Learning
- WILDS: A Benchmark of in-the-Wild Distribution Shifts
- Trusting small training dataset for supervised change detection
- Koopman-Assisted Trajectory Synthesis: A Data Augmentation Framework for Offline Imitation Learning
- RESOUND: Towards Action Recognition Without Representation Bias