Asking Experts About Harvests, Not Weights: Delphi-Elicited Rewards for Nitrogen Management Near the Yield Plateau

Adding nitrogen fertilizer raises crop yield only up to a point. Past that point the yield curve flattens, while the unused nitrogen washes into groundwater or escapes as nitrous oxide. Reinforcement learning agents trained in crop simulators already find fertilization schedules that use less nitrogen for the same yield. Each of these agents, though, optimizes a reward that adds yield revenue, fertilizer cost and a leaching penalty using coefficients the authors picked by hand. Those coefficients decide where on the flattening curve the agent stops. We propose to set them from a structured expert panel, and to do it differently from the obvious approach. Experts will not be asked "how important is leaching relative to yield?" Instead they will compare concrete simulated season outcomes, such as "10.2 t/ha with 180 kg N and 25 kg leached" against "10.6 t/ha with 260 kg N and 70 kg leached." We then fit a reward to those comparisons in which the value of extra yield shrinks and the harm of extra leaching grows along the curve.

The two methods used for elicitation are the Delphi method and the analytic hierarchy process. The Delphi method is an anonymous, multi-round survey in which panelists see the group's summarized answers and revise their own. The analytic hierarchy process (AHP) turns a matrix of pairwise "how many times better" judgments into ratio-scale scores using the matrix's leading eigenvector, and it flags self-contradictory judgment sets with a consistency ratio. The claim is that eliciting preferences over outcomes and fitting a curved reward produces agents that stop near the agronomic plateau for defensible reasons. By contrast, plugging AHP criterion weights linearly into the reward, which is the natural three-way combination, produces policies that mostly reflect how the modeler normalized each criterion's units.

Core Claim

A weight from AHP at the criterion level ("yield matters 0.55, leaching 0.30, cost 0.15") has no units. To use it in a reward, someone must first rescale yield, kilograms of nitrogen and kilograms leached onto a common range. That rescaling choice sets the exchange rate between a kilogram of grain and a kilogram of nitrate, and it quietly brings back the arbitrariness the elicitation was supposed to remove. A linear reward also assumes the exchange rate is constant. The agronomic record says it is not: the Tumochuan maize trial found a linear-plus-plateau optimum of 237.3 kg N/ha, against local practice of 300 kg (Long-term nitrogen management strategies…). Above that rate the marginal yield value of nitrogen is essentially zero while its environmental cost keeps rising.

Our alternative is to apply AHP at the level of alternatives. The alternatives are season outcomes drawn from a spread of simulator policies, and the Delphi process stabilizes the panel's comparisons. Concrete outcomes carry their own units, so no rescaling step is needed. Because the outcomes span the curve from under-fertilized to over-fertilized, the fitted reward can reveal whether experts discount yield gains near the plateau. The contribution is a method plus a falsifiable prediction: agents trained on the outcome-elicited reward will be less sensitive to arbitrary normalization and will match experts' held-out judgments better than agents trained on criterion-weighted or hand-tuned rewards.

Relation to Prior Work

The method builds on two sequential-decision papers. Learning Crop Management by Reinforcement: gym-DSSAT wraps the mechanistic DSSAT maize model as a reinforcement learning environment. Its PPO policy used about 28% less nitrogen than a 1982 expert policy at comparable yield, spread over roughly six small applications instead of three. Optimizing Nitrogen Management with Deep Reinforcement Learning and Crop Simulations adds leaching and an excess-nitrogen threshold penalty to the reward and trains DQN and SAC agents at Ames, Iowa (1999) and Gainesville, Florida (1982). Both papers scalarize their objectives with fixed constants they do not justify. This gap motivates the proposal, and we reuse their environment and agents unchanged. The second paper also found large performance drops under partial observation. That result limits us: we keep the observation space fixed across all reward conditions, so that reward design is the only variable being changed.

Only one paper in our corpus applies AHP to agriculture. GIS-Based Evaluation of Soybean Growing Areas Suitability in China derives fixed criterion weights (precipitation 0.2744, accumulated temperature 0.2657) and applies them as a linear overlay across all of China. That is the criterion-level, constant-weight use we argue against for input-response problems. We keep AHP's eigenvector scoring and consistency check but move them to the level of outcomes.

The Delphi design follows the nogaps project. That project needed three rounds before researchers and field practitioners converged on an implementable protocol, and it showed that lab-derived rules go unused when practitioners find them misaligned with field constraints. That is our reason for including extension officers and farmers alongside soil scientists and hydrologists. The known weakness of Delphi is that it measures agreement, not truth. We address this directly in one experiment by comparing the panel's implied plateau against the simulator's plateau.

Two field trials serve as external sanity anchors rather than benchmarks. The wolfberry trial found maximum yield at W1N2 but maximum nitrogen efficiency at W1N1, and recommended 150 kg N/ha with 392.40 mm of irrigation (Water and nitrogen regulation strategy for wolfberry farmland…). The Tumochuan plateau of 237.3 kg/ha is the second anchor.

Research Program

The first step builds a pool of outcomes. We train a family of SAC agents in gym-DSSAT at both sites, sweeping reward coefficients widely, and record each policy's mean season yield, total nitrogen applied and nitrate leached over many weather seeds. From this set we keep a well-spread approximation of the Pareto front, meaning the policies that no other policy beats on all three outcomes at once. We also keep a few dominated "farmer-typical" high-nitrogen outcomes, so the panel sees the region past the plateau. Each outcome is shown to panelists as a plain three-number season summary, without policy details.

The second step is the Delphi-AHP elicitation. A panel of roughly 25 people is recruited across soil science, agronomic extension, hydrology and commercial farming. Round one filters and names the outcome dimensions, using a content validity ratio to decide whether panelists want, for example, nitrous oxide shown separately. Rounds two and three present small overlapping sets of five to seven outcomes, so that each matrix needs at most 21 comparisons. This avoids the quadratic fatigue that sinks large AHP matrices. Panelists fill in 1-to-9 pairwise judgments, and any matrix with a consistency ratio of 0.10 or higher is returned to that panelist with the offending triads highlighted. Between rounds, panelists see the anonymous median and interquartile range of each outcome's score, plus justifications from outliers. The rounds stop when at least 75% of panelists fall within a predefined band of the median for each outcome, or after three rounds. We also record attrition by stakeholder group, because selective dropout could bias the result.

The third step fits the reward. AHP gives each outcome $b$ a ratio-scale score $s_b$, and the overlapping sets are chained onto one scale through their shared outcomes. We fit

$$\log s_b = f_Y(Y_b) - f_N(N_b) - f_L(L_b) + \varepsilon_b,$$

where $Y_b$, $N_b$ and $L_b$ are the outcome's yield, nitrogen applied and nitrate leached, each $f$ is a monotone spline, and $\varepsilon_b$ is residual disagreement. Diminishing returns in the panel's values show up as a concave $f_Y$ and a convex $f_L$. The quantity of interest is the implied exchange rate along the simulator's yield response curve: how many kilograms of grain the panel would give up to avoid one more kilogram of leaching. The linear reward forces this rate to be constant. We fit one pooled reward and one per stakeholder group, so the analysis can show whose preferences decide where the agent stops.

The fourth step retrains agents. Using the same SAC and DQN configurations as Optimizing Nitrogen Management…, we train on the fitted reward, applied at season end to the season's totals, and compare against the baselines below.

Baselines

The naive reference points are the null policy (no fertilizer) and the historical expert policy shipped with gym-DSSAT. Both are run on the same weather seeds as every other condition. They anchor the scale of yield, nitrogen and leaching, but they are not competitors on the question of reward design.

The strongest existing method is the hand-tuned multi-term reward of Optimizing Nitrogen Management with Deep Reinforcement Learning and Crop Simulations. At Ames it reported 21,711.8 kg/ha yield with 240 kg/ha N, against 21,709.5 kg/ha at 280 kg/ha for the best baseline. In Florida drought conditions it reported 6,310.8 kg/ha yield against 5,190.4, with leaching of 33 kg/ha against 97. We reimplement it with identical agents, observation space, training episodes and seeds, and first confirm that we reproduce numbers close to these before any comparison.

Matching or beating those yields is not the test, because a different reward legitimately chooses a different trade-off. Our reward counts as beating the baseline if, on held-out panel judgments, it ranks outcomes more accurately than the hand-tuned reward does. The held-out judgments are 20% of comparisons, withheld from fitting, together with a fresh pairwise ranking of the trained agents' own outcomes in a final validation round. A second requirement is that the resulting policy is Pareto-competitive, meaning no baseline policy beats it on all three outcomes.

The key ablation is criterion-level AHP with a linear reward: the obvious combination of the three directions. The same panel gives pairwise judgments on yield, cost and leaching as abstract criteria, with the same Delphi rounds and consistency filter. The weights are applied after min-max scaling to the outcome pool's ranges. This isolates the effect of eliciting over outcomes rather than criteria.

Two further ablations isolate the remaining ingredients. The first keeps outcome-level comparisons but fits a linear rather than spline reward, which tests whether the curvature matters. The second uses only round-two judgments with no feedback, which tests whether the Delphi iteration matters. All conditions share the same training compute and the same hyperparameter search budget.

Why not just…?

Why not just have the panel pick a policy from the Pareto front, with no reward fitting at all? This is the most serious threat, and for a single site in a single season it may be enough. Its limitation is that a picked policy does not transfer: the Ames front and the Gainesville drought front hold different policies, and choosing from each needs a new panel round. We test this directly. The panel picks from the Ames front only. We then compare two ways of carrying that choice to Gainesville: the reward coefficients that generated the picked Ames policy, and our fitted outcome reward. The winner is whichever one produces a Gainesville outcome ranked higher in a final round in which the panel sees Gainesville outcomes. If post-hoc picking transfers as well, the reward-fitting step is unnecessary and we will report that.

Why not just set the reward from economics, with grain price, fertilizer price and a regulatory leaching fine? Where a real price exists for nitrate this is cleaner, and we include it as a condition wherever a defensible fine can be stated. In most watersheds, though, the leaching price is exactly the contested quantity that farmers and hydrologists disagree on. Prices also give a linear reward, so the curvature ablation speaks to this alternative as well. If the spline reward adds no held-out predictive power over a linear one, the price-based reward is as good and should be preferred for its simplicity.

Why not just ask experts for direct numerical weights, or use the fewer comparisons of the Best-Worst Method? Direct weighting avoids AHP's comparison burden, but it still operates at the criterion level and inherits the same normalization problem. The criterion-level ablation, run once with direct rating and once with AHP, shows whether the elicitation format matters or only the level of what is being compared. We expect the level to dominate. If direct rating over outcomes predicts held-out judgments as well as AHP does, we will drop AHP in favor of the simpler format.

Decisive Checks

The first check is normalization sensitivity. We rescale the criterion ranges used by the criterion-level AHP baseline by up to 50% in each direction and measure the change in seasonal nitrogen applied by the retrained agents. The outcome-level reward needs no such scaling, so it has no corresponding knob. The prediction is that the criterion-level policy's nitrogen use swings by tens of kilograms per hectare under rescaling, which would show that its "expert weights" were mostly the modeler's normalization choice.

The second check is where the agent stops. On the simulator's own yield response curve at each site, we compute the plateau rate and compare the nitrogen used by each trained agent against it. Our reward should stop at or just below the plateau. The linear rewards may overshoot or undershoot depending on the arbitrary exchange rate. We also compare the panel's implied plateau, the point where $f_Y$ flattens relative to $f_N$, against the simulator's plateau. A large gap would be concrete evidence of the consensus-versus-truth weakness of Delphi, and would tell us that the panel should be shown response curves before judging.

The third check tests whether the fitted reward makes sensible choices beyond the maize simulator. We present the Tumochuan and wolfberry treatment outcomes as outcome bundles and ask which treatment the fitted reward prefers. For wolfberry, where yield and efficiency optima diverge, the test is whether it lands nearer the trial's recommended 150 kg N/ha than a yield-maximizing rule does. This is a qualitative check: if the reward prefers W1N2 or 300 kg N at Tumochuan, the elicited preferences are yield-dominated, and we will report it as such.