Satellite crop maps work well for common crops in the regions where models were trained. They fail where food-security monitoring most needs them: rare crops in new regions. Two problems drive this failure, and they reinforce each other. A rare crop is often grown in only a few training areas, so a model can learn to recognize it by the "look" of those areas, meaning their atmosphere, sensor calibration, local climate and planting calendar, rather than by the crop's own seasonal growth pattern. In a new region that look changes and the rare crop vanishes into the dominant classes. The obvious fix is to give the model more surrounding context, such as what the neighboring fields are doing on the same dates. That runs into the transformer's context limit. A season-long image stack of a whole scene is far too many tokens to process at once, so current systems average each field into one short time series and classify fields one at a time.
We propose to treat a satellite scene the way computational pathology treats a whole slide. Every field in a scene shares the same nuisance "style," just as every patch on a slide shares one stain. The proposal has two parts. First, encode all fields in a scene jointly with a long-context transformer and pool each field's representation only at the end. This is late chunking, borrowed from long-document retrieval. Second, train with contrastive negatives drawn from other fields in the same scene. Because those negatives share the scene's style, style cannot help tell them apart, and the representation is pushed to encode crop identity instead. If the idea holds, it should help rare crops most under cross-region transfer. The note also specifies the cheap controls that could show the joint encoder is unnecessary.
Core Claim
The claim is that within-scene contrast together with joint scene encoding is a better unit of training for long-tailed crop mapping under geographic shift than per-field encoding with either loss reweighting or global feature alignment. Shared scene style is a shortcut that is strongest for tail crops. Contrasting a field against its same-scene neighbors removes that shortcut without discarding the environmental information that geographic generalization needs. Joint encoding then lets each field's growth curve be read relative to its neighbors' curves on the same acquisition dates. This should absorb nonlinear shifts such as a delayed regional planting calendar, which a fixed additive correction cannot capture.
Two pathology results motivate the analogy. MixUp-MIL found that interpolating patch features across slides hurts accuracy because it blends different staining environments. Interpolating within one slide helped: accuracy rose from about 0.72 to 0.78 on frozen sections and from about 0.81 to 0.84 on paraffin sections. "Stay inside the shared environment" is therefore a demonstrated principle, not a guess. HistAuGAN showed that unaugmented features cluster by hospital, with mean local diversity of 0.20, and that style-randomizing augmentation breaks this clustering, raising it to 0.85. Representations can silently encode site identity. We reuse the diagnostic rather than the generator.
One result constrains the design. ARCDoG found that forcing crop classifiers to be invariant to climate degrades geographic generalization. Keeping correlation with bioclimatic variables improved regional accuracy by 4% to 11%, including a rise from 0.484 to 0.600 in the US Southeast. This rules out adversarial scene-invariance. Our negatives do not erase environmental information globally. They only require that fields sharing an environment be distinguishable. The shared context remains available to every field through joint attention.
The context mechanism comes from Context is Gold. That work encodes a whole document with an 8,192-token encoder and pools passages only at the final layer. It adds in-sequence negative training, which contrasts a passage against other passages from the same document so they do not collapse to near-identical vectors. On ConTEB, a benchmark of retrieval that requires surrounding context, this raised nDCG@10 from 52.0 to 75.6. That work is text-only and does not study label skew or domain shift. Our contribution is to show that, for geospatial data, same-sequence negatives serve a second purpose: they cancel batch effects. The prediction is that this matters most for the label tail.
Research Program
The scene is a spatial tile observed over one growing season. Each field in it becomes a short run of tokens, one per acquisition date. Each token holds the field's mean spectral bands for that date, plus a date embedding and the field's relative position in the tile. A scene with roughly 200 fields and 25 cloud-filtered dates comes to about 5,000 tokens. That fits inside an 8k-token encoder with full attention. Tokenizing per pixel would not fit, and this budget arithmetic is why field-level tokens are the chosen granularity. Missing dates are simply absent tokens rather than interpolated values. This avoids the linear gap-filling that CropSTGAN relies on.
The encoder is pretrained with masked reconstruction of field-date tokens, following the label-free temporal pretraining that Reuss et al. found most helpful for rare crops. After the final layer, each field's tokens are pooled into one field embedding. Fine-tuning combines two losses. The first is a supervised classification loss. The second is an in-scene contrastive loss: for each labeled field, positives are other fields of the same crop in other scenes, and negatives are fields of different crops in the same scene. For the supervised loss we use the class-wise ranking with a floor on negative gradients from Distributionally Robust Loss. That paper showed that ranking within a class across a batch avoids the "co-occurrence trap," where common labels are predicted because they co-occur with rare ones. This trap has a direct spatial analogue: rare crops sit next to dominant grassland.
The analysis component asks whether the mechanism is the one claimed. We measure how well a linear probe on field embeddings predicts the scene or region a field came from. This is the remote-sensing counterpart of HistAuGAN's hospital-clustering diagnostic. We expect in-scene negatives to lower this predictability while keeping crop accuracy. We then measure scene leakage separately for head and tail crops. The hypothesis predicts that the reduction is larger for tail crops, because their training examples are concentrated in fewer scenes.
The primary data is EuroCropsML, built from Latvian, Estonian and Portuguese parcel registries with the hierarchical HCAT crop taxonomy. We reconstruct scenes by grouping parcels by location into tiles. The main risk is that the released benchmark exposes per-parcel time series without enough geometry to do this. If so, we rebuild scenes from the underlying public parcel registries for the same countries and years. Transfer settings are Latvia→Estonia, where the climates are similar, and Latvia/Estonia→Portugal, the Mediterranean shift where Reuss et al. report a large penalty and little crop overlap. "Tail" means classes with at most 20 labeled target parcels, matching that benchmark's few-shot regime. CropHarvest contains point samples without scene structure. It cannot test joint encoding, so we use it only as an unlabeled pretraining source.
Baselines
The naive reference point is direct transfer. A per-field transformer of matched size is pretrained the same way, trained on source parcels with plain cross-entropy, and applied unchanged to the target. This is the setting in which Reuss et al. observe collapse onto dominant classes. It anchors how much any method gains on the tail.
The strongest existing method on this exact benchmark is the EuroCropsML suite of few-shot transfer approaches. The relevant ones are the CrossPRESTO-XTS self-supervised encoder and MAML. The only reported figure in our file is MAML's peak accuracy of about 65% on 500-shot Latvia→Estonia tasks (Reuss et al.). For CrossPRESTO-XTS tail-class accuracy at $k \le 20$, no reported result was found on Lacuna, so we rerun it. We will reproduce both with the authors' protocol, then give them the same pretraining data and the same tuning budget as our model. That budget is 30 hyperparameter trials and five seeds per setting. Beating them means a higher tail-class macro-F1 on Portugal with confidence intervals that do not overlap across seeds, while overall accuracy stays within one point.
The strongest "cheap correction" competitor is feature-and-prior shift adjustment, FPSA. It recenters target features by expected class means and reweights posteriors with regional crop statistics. It cut misclassification by 21.9% on average in Occitanie and by 6.6%, 28.4% and 42.7% in three Kenyan regions, using linear classifiers on different data. We apply it on top of the per-field baseline's embeddings, using regional crop shares computed from the target registry, which gives it an information advantage. Beating it means outperforming it without access to those statistics.
Two ablations isolate the key ingredients. The first keeps joint scene encoding but draws contrastive negatives at random from the whole batch instead of from the same scene. The second keeps in-scene negatives but encodes fields independently, building the negative set by scene membership alone. The full method should beat both, and on the tail the first ablation should lose more than the second.
Why not just…?
Why not just reweight or oversample rare crops? Frequency fixes leave the shortcut intact. A rare crop seen mostly in two Latvian tiles is still recognizable by those tiles' look, however heavily it is weighted. Shin et al. report negligible gains from such adjustments in large imbalanced label sets. Distributionally Robust Loss shows that ranking structure raises COCO-LT tail mAP from 42.88% to 53.76%. Its evidence is from images, though, so we include inverse-frequency reweighting as a run on our data rather than assume it fails.
Why not just standardize each scene, subtracting the scene-mean spectra per date before a per-field encoder? This is the most serious threat. It is cheap, needs no long context, and removes any purely additive scene style, much as FPSA's feature shift does. Our claim rests on scene effects being nonlinear, for example a two-week planting delay that reshapes a crop's growth curve rather than offsetting it. Per-scene standardization is therefore a full baseline, run with the same budget. If it matches the full method on Portugal tail F1, the long-context part of the proposal is unjustified and we will report it that way. In that case we would keep only in-scene negatives on top of standardization.
Why not just append a scene summary token, the mean of neighbors' features, to each field's independent encoding? This gives each field coarse context at almost no cost. What it lacks is date-aligned, field-to-field comparison, such as "this field greened up after the barley fields next to it." We run this variant. If it closes most of the gap to full joint encoding, the honest conclusion is that scene context matters but full attention does not.
Why not just align source and target globally with a CycleGAN-style mapper? CropSTGAN does this. It reports 84.91% overall accuracy when transferring from the USA to China, but it assumes the two regions share the same crops, and it has no mechanism for the tail. Portugal's small crop overlap with the Baltic countries breaks that assumption by construction. ARCDoG's finding that invariance to the environment can hurt also suggests that global alignment is the wrong goal here.
Proposed Experiments
The decisive test is tail-class macro-F1 on Latvia/Estonia→Portugal across the full method, the four baselines and the two ablations, with five seeds each. Overall accuracy and head-class F1 are reported alongside to expose any trade-off. The hypothesis is supported only if three things hold. The full method beats per-scene standardization and the summary-token variant on the tail. The ablation with random negatives loses most of that tail gain. And the scene-predictability probe drops more for tail than for head crops.
A shuffled-context control separates per-scene co-observation from regional crop priors. It feeds each target scene's context with fields from a different scene of the same region and the same dates. If accuracy survives the shuffle, the model is mostly learning which crops are common locally, an implicit prior shift in FPSA's sense. That would not support the claim that neighbors' simultaneous growth curves are informative.
An early-season check truncates target scenes to their first half of acquisition dates. The proposal predicts the advantage over per-field encoding is largest when the regional planting calendar is shifted relative to the source, because relative timing is visible before absolute crop signatures diverge. We test this by comparing gains on Portugal with gains on the climatically similar Estonia transfer.
Finally, a small probe tests whether the principle runs in the other direction. On Camelyon17, which has slides from five hospitals, we compare patch embeddings trained with negatives drawn from the same slide against embeddings trained with random negatives, and measure hospital clustering. The reference values are HistAuGAN's mean local diversity of 0.20 without augmentation and 0.85 with it. For tumor-classification PR-AUC under this protocol, no reported result was found on Lacuna. If within-slide negatives reduce site clustering comparably without a generator, the "scene as slide" analogy is supported in both directions. If they do not, the crop results stand on their own, and the cross-domain claim is dropped.
References
- MixUp-MIL: Novel Data Augmentation for Multiple Instance Learning and a Study on Thyroid Cancer Diagnosis
- Structure-Preserving Multi-domain Stain Color Augmentation Using Style-Transfer with Disentangled Representations
- Activation Regression for Continuous Domain Generalization with Applications to Crop Classification
- Context is Gold to find the Gold Passage: Evaluating and Training Contextual Document Embeddings
- Cross Domain Early Crop Mapping Using CropSTGAN
- Meta-learning For Few-Shot Time Series Crop Type Classification: A Benchmark On The EuroCropsML Dataset
- Distributionally Robust Loss for Long-Tailed Multi-label Image Classification
- CropHarvest: A global dataset for crop-type classification
- Two shifts for crop mapping: Leveraging aggregate crop statistics to improve satellite-based maps in new regions
- Approach for Video Classification with Multi-label on YouTube-8M Dataset