Greener Is Not Better: Auditing Canopy-Index Proxies for Nitrogen Strategies

Farmers and agronomic models increasingly judge nitrogen fertilizer strategies by how the crop looks from above. A drone or satellite measures how green the canopy is, and greener plots are read as better managed. Generative audio has already learned, at some cost, what happens when systems are ranked by a cheap signal that only partly tracks what matters. Metrics that match surface statistics reward outputs that listeners reject and penalize outputs they prefer. That field's answer was to meta-evaluate its metrics: score many systems with both the cheap metric and expensive human judgment, measure how well the two rankings agree, and correct the cheap metric with a small trusted sample. This proposal applies the same discipline to nitrogen management. The cheap metric is canopy greenness. The expensive truth is nutrient use efficiency, meaning grain gained per unit of nitrogen applied minus the nitrogen lost to leaching.

The concrete claim is that canopy-index proxies are biased in a predictable direction. They saturate under heavy fertilization and cannot see nitrate leaving through the soil. As a result they rank aggressive strategies too high, most of all in wet years. We expect this bias to be correctable with a few dozen expensive ground-truth measurements per evaluation if those measurements go to the plots where the proxy is least trustworthy. We will measure the divergence where truth is fully known (crop simulators), check its yield-side signature on real trial data, and test a prediction-powered evaluator that combines cheap canopy scores with sparse ground truth to rank strategies correctly.

Core Claim

Two facts in the literature point the same way without being connected. Detecting In-Season Crop Nitrogen Stress of Corn for Field Trials Using UAV- and CubeSat-Based Multispectral Sensing shows that the Chlorophyll Index Green, a band ratio of near-infrared to green reflectance, tracks late-season leaf nitrogen closely (r = 0.951) across 28 treatments. It also notes that heavy rainfall complicates interpretation. Optimizing Nitrogen Management with Deep Reinforcement Learning and Crop Simulations shows that in a Florida drought scenario the high-nitrogen baseline leached 97 kg/ha of nitrate and yielded 5,190.4 kg/ha, while an adaptive policy leached 33 kg/ha and yielded 6,310.8 kg/ha.

A metric that correlates with leaf nitrogen across treatments is not the same as a metric that ranks strategies by efficiency. Leaf nitrogen can stay high under luxury uptake, where the plant absorbs more than raises yield, while surplus nitrate leaves the root zone. The audio literature documents the identical structure. A Benchmarking Initiative for Audio-domain Music Generation using the FreeSound Loop Dataset finds that distribution metrics reward correct drum textures with chaotic timing. SongEval quantifies the gap: existing aesthetic metrics reach r = 0.61–0.66 with human ratings, against 0.912 for a learned evaluator. We borrow the methodology from these papers, not their models. The new object is a policy-level meta-evaluation of agronomic proxies, with an explicit hypothesis about the direction and conditions of their bias.

Research Program

The first component is a testbed where the truth is known. We run gym-DSSAT, a crop model wrapped as a reinforcement-learning environment, and CropGym, which uses the LINTUL-3 crop model over Dutch weather from 1983–2016 as studied in Analyzing Reinforcement Learning Algorithms for Nitrogen Fertilizer Management in Simulated Crop Growth. We build a pool of roughly fifty nitrogen strategies that spans the space of behavior. The pool includes fixed-rate rules at several totals, split schedules with sidedress (a mid-season top-up applied beside the rows), the weather-adaptive reduction logic of CLEANS from Mitigating nitrogen losses with almost no crop yield penalty during extremely wet years, and reinforcement-learning policies of varying aggressiveness, including DQN agents prone to the over-fertilization "overestimation trap" and the more cautious Dueling DDQN.

For each strategy and weather year, the simulator gives the true efficiency-adjusted return $J$: yield value minus fertilizer cost minus a leaching penalty. To produce what a sensor would see, we pass simulated leaf area and leaf chlorophyll through the PROSAIL canopy reflectance model, the radiative transfer model used in A combined model of shoot phosphorus uptake based on sparse data and active learning algorithm. From the resulting reflectance we compute the Chlorophyll Index Green and NDVI, then add sensor noise. This gives each strategy a proxy score with a physical basis instead of a hand-made one.

The second component is the meta-evaluation, run exactly as the audio work does it. We compute rank agreement between proxy scores and $J$ across the strategy pool, measured with Kendall's tau. We also compute selection regret, the true-return gap between the strategy the proxy picks and the truly best one. Both are stratified by weather regime. The hypothesis is directional. Divergence should concentrate among high-input strategies, where reflectance saturates, and in extremely wet years. The PEPIC analysis behind CLEANS identifies wet years as the drivers of extreme nitrogen loss, and in those years greenness should stay high while nitrate leaches.

The third component is the correction. We use prediction-powered inference, the debiasing method that Scalable Vision-Guided Crop Yield Estimation applied to photo-based yield estimates. For strategy $\pi$, suppose we have $N$ plots scored only by a learned proxy predictor $f$ and $n$ plots with expensive ground truth. The estimate of mean return is

$$\hat\theta_\pi = \frac{1}{N}\sum_{i=1}^{N} f(x_i) + \frac{1}{n}\sum_{j=1}^{n}\big(J_j - f(x_j)\big),$$

where $x$ is the canopy observation and weather record for a plot and $J_j$ is its measured return. The second term removes the proxy's average bias, so the estimate stays unbiased even when $f$ is wrong. A better $f$ only narrows the confidence interval.

Li et al. used this method for one static yield total. We extend it in two ways. First, we estimate differences between strategies, which is what ranking needs. Second, we choose where the $n$ measurements go, steering them toward high-input plots and wet years, where we predict the proxy residual has the most variance. In simulation the "measurement" of $J$ is free, but we cap $n$ at a budget comparable to what soil cores and lysimeters (instruments that collect water draining below the root zone) could realistically support.

The fourth component asks whether the bias matters for optimization. We train policies in gym-DSSAT under three regimes: reward computed from the canopy proxy, reward from the corrected evaluator, and true reward as an oracle. We then measure true leaching and return. If proxy-reward agents learn to over-fertilize for greenness, that is the agronomic version of reward hacking, the behavior the music reward-model work in CMI-RewardBench warns about.

Baselines

The naive reference point is static practice: the fixed high-nitrogen schedules that gym-DSSAT and Wu et al. use as baselines. For evaluation itself, the naive reference is ranking strategies by raw end-of-season canopy index with no ground truth. Every corrected method must beat this proxy-only ranking on Kendall tau and selection regret over the same strategy pool and weather years.

The strongest existing competitor is supervised calibration. A gradient-boosted regressor maps canopy index time series, applied nitrogen, and weather to $J$, is trained on the same $n$ labeled plots, and is then used alone. It is the strongest competitor because the canopy-index literature already shows high correlations with plant nitrogen, so a good regression might absorb most of the bias. It gets the same labeled plots, the same features, and the same hyperparameter search budget as the proxy model $f$ inside our estimator. Our method beats it if, at equal $n$, it gives lower selection regret and confidence intervals that actually reach nominal coverage on held-out weather years. Calibration-only point estimates come with no coverage guarantee under weather shift.

A second strong baseline adds a cheap soil test as an input to that regressor. In simulation this is a single early-season soil nitrate reading drawn from the model state. It tests whether the bias is just missing information that one soil measurement supplies.

Two ablations isolate the key ingredients. Prediction-powered inference with randomly placed labels separates the value of debiasing from the value of targeted measurement. Targeted labels with plain regression instead of debiasing separates measurement placement from the bias-correction term. For policy training, the oracle true-reward agent sets the ceiling. Wu et al. report 33 kg/ha leaching and 6,310.8 kg/ha yield for their adaptive policy in the Florida scenario under full observation. The corrected-reward agent should approach these numbers, and the proxy-reward agent is predicted to fall well short on leaching.

Why not just…?

Why not just use the simulator's true reward and skip proxies? In simulation that is exactly the oracle, and we use it. But the reason to study proxies is that real fields do not report leaching. Wu et al. already show that their agents collapse when restricted to realistic partial observations. The simulator here is a measuring instrument for proxy bias, not the deployment target.

Why not just calibrate the canopy index by regression on the labeled plots? This is the real threat. If weather shift between training and test years is mild, regression may match our estimator on point accuracy. The calibration baseline above is designed to settle the question. We expect the debiasing term to matter most in held-out wet years, where a regression fit mostly on normal years extrapolates. If it does not matter there, we will report that the contribution reduces to label placement plus the meta-evaluation finding itself.

Why not just use a more cautious learning algorithm such as Dueling DDQN? Vogt and Rosman show that Dueling DDQN avoids over-application under the true simulated reward, and they report no numerical comparison. Caution about value estimates does not help when the reward signal itself prefers greenness. The proxy-reward training arm will show whether a cautious learner still drifts toward over-fertilization once its reward is the canopy index.

Why not just add the CLEANS wet-year rule? CLEANS cuts inputs from precipitation anomalies and achieves 21–46% lower extreme losses with under 3% yield penalty. It is a strategy, not an evaluator, so it enters our pool. If our hypothesis holds, the proxy will rank CLEANS below its true position in wet years, which is a direct, checkable prediction.

Proposed Experiments

The decisive simulation check has three parts. First, compute Kendall tau and selection regret between proxy and truth over the strategy pool, stratified by input level and by wet versus normal year, across held-out CropGym years (1984, 1994, 2004, 2014, 2019, 2020) and the gym-DSSAT Florida and Iowa scenarios. The hypothesis fails if divergence is uniform rather than concentrated in high-input, wet-year cells. Second, plot selection regret and interval coverage against label budget $n$ for the proxy-only ranking, calibration, calibration with a soil test, randomly labeled debiasing, and targeted debiasing. Third, compare true leaching and return for agents trained on proxy, corrected, and true rewards.

The real-data check uses the Central Illinois 2017 trial from Cai et al., with 28 nitrogen treatments, UAV and CubeSat indices, leaf nitrogen, and yield. We test whether canopy-index rankings of treatments diverge from yield-per-nitrogen rankings specifically among the highest-rate treatments, which is the saturation signature. That trial has no leaching measurements, so it can confirm only the yield-efficiency half of the claim. No reported result found on Lacuna for field-measured leaching paired with canopy indices. A field campaign with lysimeters would be the natural follow-up.

The phosphorus dataset from Wang et al., where regression on spectra already failed for stem uptake, serves as a second nutrient to test whether the directional bias generalizes beyond nitrogen. For context on effect sizes, Li et al. report that debiasing increased effective sample size by 73% for rice and 12–23% for maize. Our targeted variant should exceed the random-placement gain on high-input strategies if measurement placement matters.