Oscillate or Progress: Telling Farmland Conversion from Crop Cycles in Satellite Time Series

Cropland disappears under cities, fish ponds and abandoned scrub at rates that matter for food security, yet the satellite record that tracks this loss is crowded with change that is not loss. Fields green up, get harvested, lie fallow for a season, switch crops and dry out in droughts. A detector that compares two images will flag all of that. Computer vision has faced the same problem in video, where swaying trees and rippling water look like motion. One old and simple answer there is that background clutter oscillates while real objects keep moving in one direction. We propose to carry that principle into multi-year satellite series. The crop cycle becomes the "dynamic background" that we model and keep, rather than a nuisance to be removed. Conversion is then the moment a pixel stops oscillating around its own seasonal pattern and starts drifting steadily in one direction. The direction of that drift tells us what the land is becoming.

Concretely, we learn, for each location, what it normally looks like at each point in the season. We measure how far each new observation departs from that expectation, and we add up departures only while they keep pointing the same way, resetting the sum whenever the direction reverses. A fallow year or a crop rotation produces departures that reverse within a season. Paving, digging a pond or gradual abandonment produces departures that persist and accumulate. The final accumulated direction is then matched against a few destination types, such as built-up land, water, orchard and bare or abandoned ground. The output is not only "changed" but "cropland became X, starting around date t."

Core Claim

Gradual, multi-stage conversions are the hard cases in agricultural land conversion: peri-urban plots that are first fenced, then graded, then built on, and fields that slide into abandonment over several seasons. Confusing these with fallowing and rotation is the dominant error. The open-problems literature names exactly this confusion as unresolved (Chengdu cropland abandonment study; marginal land extraction). We claim that a direction-consistent accumulation of season-normalized residuals, computed in a learned image-embedding space, separates these cases better than three alternatives: detectors that look for a single sharp break point, bi-temporal deep detectors fed carefully matched image pairs, and scalar vegetation-index anomaly tests. Because the accumulated residual has a direction, the same signal also yields a from-to label without a separate classifier trained on the full transition matrix.

Relation to Prior Work

The mechanism comes from Detecting Salient Motion by Accumulating Directionally-Consistent Flow. That work sums optical flow over video frames and resets a pixel's score to zero when its motion reverses. This suppresses foliage and lets it detect targets moving as slowly as 0.6 pixels per frame, with peak clutter salience of 55 against 140 for the true target. It assumes dense, regularly sampled frames with small displacements. Satellite series are irregular and cloud-gapped, and the "motion" here is a change in appearance, not in position. We therefore accumulate residuals from a seasonal expectation rather than raw flow, and weight each step by the elapsed time.

The seasonal expectation draws on two background-modeling ideas. Incorporating Natural Variation into Time Series-Based Land Cover Change Detection normalizes yearly vegetation change by each pixel's historical variability. This raised precision in variable shrublands from 0.01 to about 0.23. That result motivates per-location normalization. Their method works on a single vegetation index, targets abrupt drops such as fire, and explicitly counts agricultural rotation as a false positive, which is the case we must handle. Background subtraction in highly dynamic scenes models video backgrounds as dynamic textures, meaning appearance that changes according to a learned linear dynamical system. It shows that modeling the background's own dynamics beats static mixture models in snow and surf. We borrow the stance that the background has dynamics worth modeling, but we use a cheaper per-location seasonal model to avoid the fitting cost that paper reports.

The closest prior work is OPTIMUS. It learns without labels whether the frames of a satellite series can be put in temporal order: they cannot under cyclical variation, and they can after a persistent change. It locates a "pivot" where its predictions flip and reports 87.6% AUROC on 300 annotated Sentinel-2 series. OPTIMUS is binary and assumes a distinct pivot. Our proposal differs on both counts: accumulation needs no single break, and the accumulated direction carries semantics. We reuse OPTIMUS's unlabeled data and, potentially, its encoder, so that any gain cannot be credited to more pretraining.

The bi-temporal deep detectors constrain the design. BIT, SEED and ChangeViT compare two dates. BIT uses token-level attention, SEED swaps features between the two branches instead of subtracting them, and ChangeViT adds a detail branch to a plain vision transformer. The pseudo-change suppressors FeaSpect, CDGAN and DASNet treat vegetation shifts as style noise to be erased. For farmland that design is self-defeating, because erasing vegetation dynamics erases the signal. Semantic change detection on SECOND reaches only 15.5–16.5 SeK, which shows that learning a full from-to matrix from pairs is hard. The asymmetric one-way character of conversion motivates a lighter route. Xi'an land-use study reports a state index near 1.0 for built-up land, meaning paved land essentially never reverts.

Research Program

We first train an encoder that maps each Sentinel-2 observation of a small patch to an embedding. Training is self-supervised on the OPTIMUS unlabeled series, with the objective of predicting a patch's embedding at a given day of year from the same patch's observations in earlier years. This forces the encoder to represent the recurring seasonal trajectory. Its prediction gives a seasonal expectation $\mu_t$, and the spread of recent prediction errors gives a scale $\sigma$. The residual is $r_t = (z_t - \mu_t)/\sigma$, where $z_t$ is the observed embedding.

The accumulator follows the directional-flow rule:

$$S_t = \mathbb{1}\left[\cos(r_t, D_{t-1}) > 0\right]\,\big(S_{t-1} + \Delta_t \lVert r_t \rVert\big), \qquad D_t = \mathbb{1}\left[\cos(r_t, D_{t-1}) > 0\right] D_{t-1} + r_t .$$

Here $S_t$ is the accumulated persistence score, $D_t$ is the accumulated residual direction, and $\Delta_t$ is the time since the previous cloud-free observation. Missing dates are skipped rather than interpolated. A fallow season produces a residual that reverses when cropping resumes, which resets $S$. Conversion keeps $\cos > 0$ and lets $S$ grow, whether the change is abrupt or gradual.

We add one asymmetric prior. Once $D_t$ aligns with the built-up prototype beyond a threshold, isolated reversals toward vegetation are treated as observation noise, such as cloud shadow or misregistration, instead of resets. This encodes the one-way behavior of impervious surfaces. Abandonment gets no such protection, since it can genuinely revert.

The destination is read off by comparing $D_t$ with prototype directions estimated from a few dozen labeled examples per destination class.

The second deliverable is an evaluation set, because none in the file contains the hard negatives we care about. We will annotate Sentinel-2 parcel time series from 2016 to 2023 in the regions the conversion literature studies. These are peri-urban Xi'an, the Tianfu New Area near Chengdu where abandonment has been mapped, the Hulin black-soil zone, Jiangsu counties, and the Khulna coast where cropland turns into aquaculture. Each parcel is labeled stable cropping, rotation, fallow, abandonment, or conversion to built-up, water/aquaculture, or orchard, with an onset date. Gradual and abrupt conversions are tagged separately.

Baselines

The naive reference is yearly delta: the difference in peak-season NDVI between consecutive years with a tuned threshold. Its known failure in variable vegetation, 0.01 precision in shrublands (Mithal et al.), sets the floor.

The strongest classical competitor is a multi-band version of the variability-normalized test from the same paper. It fits a per-pixel seasonal curve to all Sentinel-2 bands and flags normalized residuals above a threshold, giving a "nearest neighbor of our idea without learning or direction." The file reports approximately 0.23 precision for the scalar version in shrublands; there is no reported result on Lacuna for cropland.

The strongest learned competitor is OPTIMUS, run on our parcels using its released training recipe and the same unlabeled pretraining data. Its reported figures on its own benchmark are 87.6% AUROC and 0.760 F1 (OPTIMUS). For from-to labels we pair it with a classifier applied to the post-pivot images, trained on the same few labels our prototypes use.

The bi-temporal competitors are SEED and ChangeViT, fine-tuned on PX-CLCD and CLCD respectively and applied to same-season anniversary image pairs chosen to minimize phenology mismatch. SEED reports 86.25% IoU on LEVIR-CD and 70.91% on SYSU-CD. There is no reported result on Lacuna for PX-CLCD or CLCD numbers.

All methods get equal hyperparameter search budgets on a held-out region. Beating the field means two things. First, a lower false-alarm rate on fallow and rotation parcels at matched recall on conversions. Second, higher recall on gradual conversions than OPTIMUS at matched false-alarm rate.

Three ablations isolate the key ingredients. Removing the reset turns the method into plain accumulated anomaly. Replacing learned embeddings with raw seasonal-normalized spectra tests whether representation learning matters. Removing the asymmetric built-up prior tests whether that prior helps.

Why not just…?

Why not just fit a seasonal curve per pixel and threshold the normalized residual? This is the most serious threat. It is the reset-free, raw-spectrum ablation crossed with the classical baseline. If it matches our method on fallow false alarms and gradual-conversion recall, then the contribution is the direction-reset rule, and we will report it as a cheap classical method. Our reason to expect a gap is twofold. A single prolonged fallow year produces a large residual that a magnitude test cannot distinguish from early abandonment until the next season. And a scalar or per-band residual has no stable direction for naming the destination.

Why not just run OPTIMUS and classify the end state? OPTIMUS already handles cyclical variation well. Its pivot score assumes one identifiable break, so multi-stage encroachment is where it should lose. Our gradual-versus-abrupt split reports this directly. If OPTIMUS-plus-classifier equals us on gradual cases, the accumulator adds nothing beyond semantics, and we would say so.

Why not just feed anniversary-date pairs to a strong bi-temporal detector? Matching the season removes phenology mismatch but not rotation or fallow coincidences: the same week of two years can show wheat and bare soil. Pair-based semantic labeling is also weak, as the SECOND SeK scores show (ASN). The rotation and fallow false-alarm rate for SEED and ChangeViT on our set tests this directly.

Proposed Experiments

The decisive experiment is the hard-negative test. On parcels labeled fallow, rotation and drought-stressed, we report false alarms per parcel-year for every method at the threshold that gives 80% recall on true conversions. The second test is onset-date error and detection delay on gradual conversions, which checks whether accumulation detects slow encroachment earlier than pivot-based detection. The third is from-to accuracy, reported as per-destination F1 and confusion between abandonment and built-up, compared with OPTIMUS-plus-classifier.

As a sanity check that we have not overfit to our own annotations, we also run the binary detector on the OPTIMUS 300-series benchmark and report AUROC against its 87.6%. A final transfer check trains prototypes in Chinese regions and tests on the Khulna coastal parcels, where conversion to aquaculture produces a water-like direction never seen during prototype fitting.

If the reset ablation shows no gain over plain accumulation, the central "oscillate or progress" claim is falsified for this domain. That result is still useful to report, because it would mean the crop cycle is better handled by magnitude normalization alone.