Cameras on drones, aircraft and satellites that estimate crop biomass from reflected light rarely see only the crop. Each pixel also holds bare soil, wet patches, last season's stalks and weeds, and that background changes from farm to farm. Published models that correct for this "spectral interference" report strong accuracy, but nearly all of them are scored on held-out samples from the same county, campaign and camera. Clinical prediction research has a name for this, the mirage of internal success, and has measured what happens when a frozen model is carried to an independent cohort: a COVID-19 mortality model fell from an AUC near 0.906 to 0.74 when moved from Danish to UK Biobank data Developing and validating COVID-19 adverse outcome risk prediction models from a bi-national European cohort of 5594 patients. A stroke model fell to chance (AUC 0.50) on a German database whose care setting the training data never covered Development and validation of a prognostic model predicting symptomatic hemorrhagic transformation in acute ischemic stroke at scale in the OHDSI network.
This proposal brings that discipline to crop sensing and asks a sharper question than "does it transfer?". We claim that the loss of accuracy when a frozen biomass model moves to a new region is driven mainly by how far the new region's background (soil brightness, residue fraction, moisture) lies from what the model saw during training. We also claim that this distance can be measured from unlabeled imagery of the new site, before anyone clips a single quadrat. If both hold, a model's "locality" becomes measurable. Physics-guided models, which are pretrained on simulated spectra spanning many backgrounds, should transfer intact inside their simulated range and fail at its edge. Empirical models should degrade steadily with background distance. The distance score then tells a practitioner which new fields can use the frozen model and which need local ground truth.
Core Claim
The "all models are local" argument All models are local: time to replace external validation with recurrent local validation says that transported models fail because local conditions differ, so local revalidation should replace external validation. It was made for clinical data, where nobody can describe the differences between sites from first principles. In canopy reflectance, a large part of the site-to-site difference is physical: the spectrum mixes leaf, soil and residue signals. We hypothesize that locality is therefore partly predictable.
Let $d_s$ be the background distance of target field $s$. It is the average Mahalanobis distance between background parameters estimated from that field's unlabeled imagery and the distribution of background parameters in the training data, or in the simulation range for physics-guided models. Let $\Delta_s$ be the increase in root-mean-square error (RMSE) of a frozen model on $s$ relative to its internal error. The central prediction is that $\Delta_s$ rises steadily with $d_s$ for empirical models. For physics-guided models, $\Delta_s$ stays near zero until the background leaves the simulated range, then rises sharply. The claim is falsified if $d_s$ does not rank fields by $\Delta_s$, measured as Spearman correlation across held-out fields, or if physics-guided and empirical models degrade alike.
Prior Work and Its Roles
The method comes from the radiative-transfer process-guided learning of Airborne hyperspectral imaging of cover crops through radiative transfer process-guided machine learning. That paper pretrained a long short-term memory network (a recurrent neural network) on 1,000,000 spectra from the SCOPE radiative transfer model, a physics simulator of light in canopies, varying soil moisture, residue and leaf traits. It then fine-tuned on 73 destructive quadrats from 23 central Illinois fields and reported biomass $R^2$ of 0.72 (relative RMSE 15.16%), with stable error across soil types and residue levels. Its "transferability" was measured only within one region with one sensor. We test it out of region and across sensors.
The empirical comparison comes from On-farm cereal rye biomass estimation using machine learning on images from an unmanned aerial system. It estimated cereal rye biomass on 15 northwest Ohio farms with a five-band DJI Phantom 4 Multispectral camera, six vegetation indices and gradient-boosted trees (XGBoost). On a held-out split of the same farms it reached $R^2$ 0.67 and RMSE 83.13 g/m², rising to 0.82 and 61.67 g/m² with manually measured height and canopy cover. Accuracy dropped above 200 g/m². The fact that Illinois and Ohio both measure cereal rye cover-crop biomass, with very different sensors, is what makes a real cross-cohort test possible.
Two papers constrain the design. Evaluating RGB Imaging and Multispectral Active and Hyperspectral Passive Sensing for Assessing Early Plant Vigor in Winter Wheat shows that the best way to suppress soil changes with growth stage: color thresholding works early, red-edge ratios such as 736/794 nm work later. Any distance score must therefore be computed within a growth stage, or soil exposure will dominate it. Cross-scale sensing of field-level crop residue cover: Integrating field photos, airborne hyperspectral imaging, and satellite data shows that residue has distinctive absorption at 2100–2300 nm in the shortwave infrared. That tells us residue can be inferred from airborne imagery but not from a five-band drone camera. The Ohio background estimate must therefore rely on visible and near-infrared soil-line statistics, and we report this as a limitation.
From clinical validation we borrow the protocol rather than a model. The model is frozen before any external data is opened. Calibration is reported alongside discrimination, as calibration slope and calibration-in-the-large (the mean offset between predictions and observations). Failures are traced to a missing operating regime, as the OHDSI study did for the German outpatient database. The radiomics literature reports that models overfit to scanner vendors Radiomics in prostate cancer imaging for a personalized treatment approach - current aspects of methodology and a systematic review on validated studies. This motivates treating sensor change as a separate shift from background change, so the two are not confounded.
Research Program
We build three things. The first is an external-validation testbed with two directions, Illinois to Ohio and Ohio to Illinois, for cereal rye biomass. The Illinois 356-band airborne spectra are converted to Phantom 4 Multispectral bands using that camera's spectral response functions, so models trained on either cohort can be applied to the other. This is the remote-sensing equivalent of the common data model that made the OHDSI study possible. Because it requires the two papers' field datasets, which are not public in the file, we will request them from the authors. As a fallback, and as a third truly external cohort, we will collect a new spring campaign: about 60 quadrats over at least eight fields on a contrasting soil order, flown with the same Phantom 4 Multispectral camera, with structure-from-motion canopy height.
The second is the background-distance score. For physics-guided models, we invert each target pixel's soil-reflectance, moisture and residue parameters with the SCOPE simulator. The score is that field's distance from the simulated parameter distribution, plus the fraction of pixels outside the simulation range. For empirical models, the same inverted parameters are compared with those of the training fields. No target labels are used.
The third is a recalibration-budget analysis. For each frozen model and each external field, we refit an intercept and slope using $k \in \{0, 3, 5, 10\}$ local quadrats. This lets us compare directly with the local-recalibration alternative.
Within-cohort leave-one-field-out runs, grouped by field, give a third, weaker level of transport between internal splits and full external validation. With 38 or more fields this yields enough $(d_s, \Delta_s)$ pairs to estimate the correlation, though with wide intervals. We will report bootstrap confidence intervals over fields and not over quadrats.
Baselines
The naive reference is a single soil-adjusted vegetation index (SAVI) fit by linear regression in the source cohort and frozen. SAVI had the largest influence in the Ohio models, so it is the simplest credible tool a practitioner would use. A second do-nothing comparison reports each model's internal score next to its external score, which measures the size of the mirage.
The strongest empirical competitor is XGBoost on the six indices selected by variance-inflation filtering in On-farm cereal rye biomass estimation using machine learning on images from an unmanned aerial system. Its reported independent-split result is $R^2$ 0.67 and RMSE 83.13 g/m². We also run the variant that adds structure, using drone-derived height in place of manual height, against the reported internal 0.82 and 61.67 g/m². Partial least-squares regression (PLSR) on the shared five bands is the data-driven comparison used by the physics-guided paper; that paper reports no PLSR number.
The strongest physics-guided method is the SCOPE-pretrained network of Airborne hyperspectral imaging of cover crops through radiative transfer process-guided machine learning. Its internal reference is $R^2$ 0.72 and relative RMSE 15.16%. It is retrained on the five shared bands so that sensor content matches. Direct numerical inversion of the simulator is included as a pure-physics reference.
The key ablation is the same physics-guided pipeline with the background held fixed in simulation: one soil spectrum, no residue, constant moisture. This isolates whether transport depends on simulated background variation or simply on pretraining with physics.
All models get the same source labels, the same 50-trial hyperparameter search with grouped-by-field inner validation, and the same frozen-before-testing rule. External labels are opened once. The proposal wins if the physics-guided model's external RMSE beats frozen XGBoost on fields inside its simulation range, with bootstrap intervals that do not overlap, and the fixed-background ablation does not. It must also win on the full claim: the distance score must rank external degradation with Spearman $\rho \ge 0.5$ for at least one model family. External results for any of these models: no reported result found on Lacuna.
Why not just…?
Why not just use SAVI, which already corrects for soil? SAVI uses one fixed soil-adjustment constant, and the Ohio study found that all six learners lost accuracy above 200 g/m² even with SAVI included. Its weakness in transport is that soil brightness and moisture change the correct adjustment from region to region, and that is the exact shift we measure. It is a baseline here. If it transports as well as the physics-guided model, that is a publishable negative result, and the distance score would still tell users when it fails.
Why not just do leave-one-field-out or spatially blocked cross-validation inside the pooled dataset? We do run it, but fields within one county share soil series, management and sensor, so the background distances they span are small. The clinical record shows internal resampling hiding large external drops. In our design the within-cohort and cross-cohort points lie on one distance axis, so we can show directly whether blocked cross-validation underestimates error at the distances real deployment involves.
Why not just recalibrate locally with a few quadrats, as "all models are local" recommends? This is the real threat. If an intercept-and-slope refit with five quadrats brings a frozen XGBoost model to the physics-guided model's accuracy everywhere, then physics adds little to transport. The practical conclusion would be "always recalibrate", and our contribution would narrow to the label-free score that decides when recalibration is needed. The recalibration-budget experiment settles this question directly. The physics-guided paper's evidence that it needed only 30% of the field data, where PLSR needed about 50%, suggests a smaller $k$ will suffice, but only out-of-region data can confirm that.
Why not just apply unsupervised domain adaptation, matching feature distributions between source and target imagery? Distribution matching can align background statistics, but it can also erase real differences in biomass between regions, because the targets are not observed. We include feature-mean and covariance alignment on the target imagery as an extra baseline under the same budget. It is useful only if it beats the physics-guided model inside the simulation range without harming calibration slope.
Decisive Checks
Three results decide the proposal. First, the internal-to-external RMSE gap for frozen XGBoost on the Ohio-to-Illinois and Illinois-to-Ohio transfers is the first direct measurement of the mirage in this field. Second, the comparison between the physics-guided model and the fixed-background ablation, restricted to external fields inside the simulation range, tests whether simulated background variety is what buys transport. Third, the Spearman correlation between the label-free distance $d_s$ and the observed degradation $\Delta_s$ across all held-out fields tests whether locality can be predicted. Results are also split at 200 g/m², because saturation could be mistaken for background shift. If only saturation explains the external errors, the background hypothesis fails, and we will report that.
References
- Developing and validating COVID-19 adverse outcome risk prediction models from a bi-national European cohort of 5594 patients
- Development and validation of a prognostic model predicting symptomatic hemorrhagic transformation in acute ischemic stroke at scale in the OHDSI network
- All models are local: time to replace external validation with recurrent local validation
- Airborne hyperspectral imaging of cover crops through radiative transfer process-guided machine learning
- On-farm cereal rye biomass estimation using machine learning on images from an unmanned aerial system
- Evaluating RGB Imaging and Multispectral Active and Hyperspectral Passive Sensing for Assessing Early Plant Vigor in Winter Wheat
- Cross-scale sensing of field-level crop residue cover: Integrating field photos, airborne hyperspectral imaging, and satellite data
- Radiomics in prostate cancer imaging for a personalized treatment approach - current aspects of methodology and a systematic review on validated studies