Reinforcement learning agents trained in crop simulators now beat expert fertilizer rules. They get there by applying nitrogen in small, frequent doses tuned to the weather. There is a catch: they assume the farmer can read soil nitrate at every depth, every day, for free. In real fields that measurement is slow and expensive. A drone sees the canopy, not the root zone. When agents lose that free view, their performance collapses in exactly the stressed seasons where good management matters most. This proposal treats "learning to manage nutrients while paying for soil information, over multi-year horizons" as a hard target task. It then asks a question borrowed from education research: should an agent learn this target by moving through separate, simpler stages, or by being exposed to all of its parts together from the start?
The education literature on curriculum fragmentation is the study of what goes wrong when knowledge is taught in isolated blocks, such as theory before practice or components before systems. It gives a sharp, testable prediction. Staged training in silos leaves the learner unable to put the pieces together when the stages end. Throwing the learner into the full problem without support causes overload. Mixing the parts together with scaffolding that is gradually removed does better than either. We turn that prediction into a controlled experiment on established crop-management benchmarks. The agronomy side of the pairing, integrated nutrient management, contributes something specific. It is the lesson that blended schedules beat single-dose ones, and it defines the realistic target task: multi-year, partially observed, with split applications and costly soil tests.
Core Claim
We claim the following. Suppose the target is a multi-year nitrogen management task where soil-state observations cost money. Then a single policy trained on an interleaved mixture of easier task variants will outperform two alternatives: direct training on the target, and a sequential curriculum over the same variants. The variants are shorter horizons and cheaper or free soil observation, with the mixture shifting smoothly toward the target. The sequential curriculum has hard stage boundaries, for example full observability for one year, then full observability for five years, then the costly-sensing target. We expect the advantage to show up as higher held-out-weather profit at equal nitrate leaching, fewer paid soil tests, and no performance dip at the point where privileged information disappears.
The mechanism we will test is this. An agent trained in a full-observation stage learns to read the nitrate variable directly. It never learns to infer soil nitrogen from weather, crop stage, and its own past applications. When that variable is removed, it has nothing to fall back on. Interleaving forces the policy to maintain both routes throughout training.
What is new is not the idea of curricula in reinforcement learning. It is three things together: turning costly soil sensing into an explicit action inside existing crop simulators, a head-to-head comparison of staged against integrated training on that realistic task, and a diagnostic of why one fails, carried out by probing whether the policy's internal state encodes an estimate of soil nitrate.
Grounding in Prior Work
The crop-simulator line motivates the target task and supplies the environments. Learning Long-Term Crop Management Strategies with CyclesGym wraps a multi-year agroecosystem model for reinforcement learning. It shows that agents trained on one-year horizons fail in five-year scenarios, and that a PPO policy earns roughly 30% higher average yearly profit than expert recommendations. Both results assume complete, free observation of soil state. That one-year-to-five-year failure is itself a fragmentation result: a skill learned in a short-horizon silo did not transfer. We generalize it and test whether mixing, rather than more stages, fixes it.
Optimizing Nitrogen Management with Deep Reinforcement Learning and Crop Simulations uses gym-DSSAT: a crop model turned into a Reinforcement Learning environment to train daily nitrogen policies for maize. Under drought in Florida, it reports 6,310.8 kg/ha yield against 5,190.4 kg/ha for the baseline, with nitrate leaching cut from 97 to 33 kg/ha. It also reports that performance degraded significantly under partial observability. That degradation is the problem we attack. The file gives no number for the partially observed case, so no reported result was found on Lacuna for partially observed gym-DSSAT performance, and we will measure it ourselves.
Analyzing Reinforcement Learning Algorithms for Nitrogen Fertilizer Management in Simulated Crop Growth compares DQN variants and PPO in CropGym. It finds that plain DQN and reactive heuristics over-apply nitrogen, while Dueling Double DQN learns to withhold fertilizer in dry years. This constrains our design. Value overestimation is a known failure mode here, so we build on a PPO backbone and report a dueling-DQN check.
The field-trial literature supplies the structure of the action space rather than a mechanism. Combined Controlled-Release and Common Fertilizer Application Increases Apple Productivity by Optimizing Soil Nutrient and Microbial Communities found that a 30/70 split between common and controlled-release fertilizer beat common fertilizer alone by 23.7% and 32.9% in two years. That result motivates keeping split, timed applications in the action space instead of a single seasonal dose. An Enhanced Faster R-CNN for High-Throughput Winter Wheat Spike Monitoring to Improved Yield Prediction and Water Use Efficiency shows what cheap aerial sensing can offer. It counts spikes at 91.2% [email protected] and predicts yield with R² of 0.82, but it observes only the canopy. That result justifies giving our agent free canopy-level observations and making subsurface nitrate the paid measurement.
We deliberately do not extend the simulators to organic, microbial, or tillage amendments. The file indicates these environments lack such actions, and inventing their dynamics would make any result uninterpretable.
The education papers supply the hypothesis and its failure modes, not evidence about agents. Towards connecting discrete mathematics and software engineering placed large software projects inside a prerequisite math course. It reported both the benefit of early immersion and severe workload overload without scaffolding. This is the reason our integrated condition includes easier variants instead of training on the target alone. AI Education Across the Curriculum: Design and Pilot Study of a Cross-Disciplinary Module Set reports AI literacy rising from 2.385 to 3.154 (r = 0.717) under cross-subject integration, but with N = 20. Architecture-AI Curriculum Integration: Theoretical Frameworks and Pedagogical Innovations proposes a staged three-stage framework with no quantitative evaluation. We take from these papers a contested design choice, staged versus integrated, and test its computational analog under controlled conditions. We make no claim that the agent results validate claims about human students.
Research Program
The first component is the environment change. In CyclesGym, which is the primary testbed, and in gym-DSSAT, which is used for replication, we remove soil nitrate and ammonium from the default observation. The agent can still see weather, crop stage, days since planting, its own past applications, and a small set of canopy-level crop variables that stand in for what a drone could estimate. We add a "soil test" action. It returns the current profile nitrate with a fixed delay and noise, and its monetary cost is subtracted from profit. The fertilizer action stays as a discrete set of doses applied at any step, so split and staggered schedules remain expressible. This defines the target task $T$: five-year horizon, costly sensing, profit reward with a leaching penalty.
The second component is a family of easier variants $\{T_k\}$. Each is indexed by horizon (one or five years) and by sensing cost $c$, ranging from zero, which with continuous readout reproduces the original full-observation setup, up to the target cost. The policy is always a recurrent network conditioned on a descriptor of $(\text{horizon}, c)$, so a single set of weights handles every variant. The curricula differ only in the order and proportion in which variants are sampled.
In the staged condition, variants are visited in blocks with hard switches. In the integrated condition, each training batch mixes variants. The weight on variant $k$ at training step $t$ is
$$w_k(t) \propto \exp\big(-\lambda(t)\, d_k\big),$$
where $d_k$ is a fixed difficulty score for variant $k$ (distance from the target in horizon and sensing cost), and $\lambda(t)$ rises linearly from zero to a large value over training. Early training therefore samples all variants roughly evenly, and late training concentrates on the target. The target is always present from the first step, and the scaffolding fades rather than switching off.
The third component is analysis. At fixed checkpoints we train a linear probe to predict true root-zone nitrate from the policy's hidden state. We expect that staged agents' probes stay poor until the final stage and that performance dips at each switch, while integrated agents' probes improve steadily. We also track how many soil tests each policy buys, and in which weather years, to check whether it learns to pay for information when information matters, for example after heavy rain.
Baselines
The naive reference points are the expert fertilizer schedules shipped with CyclesGym and gym-DSSAT, plus a null policy that applies no nitrogen. They are evaluated on the same held-out weather years. The expert rule needs no soil tests, so it is the fair benchmark for any learned policy that pays for information.
The strongest reported learned result is the full-observation PPO agent of Learning Long-Term Crop Management Strategies with CyclesGym, which reports approximately 30% higher average yearly profit than expert recommendations over five years. We retrain it with the released setup (train on 1980–2005 weather, test on 2006–2015). It sees nitrate for free, so we report it as an upper-bound oracle, not a competitor to beat. The success measure is the fraction of the oracle-over-expert profit gap that each costly-sensing agent recovers.
The strongest competitor that faces the same costly-sensing task is PPO trained directly on the target. It uses the same recurrent architecture, the same total environment steps, and the same hyperparameter search budget as the curricula. The search is run separately for each condition with an equal number of trials.
The second competitor is an asymmetric actor-critic, in which the value function sees the true soil state during training but the policy does not. This is a standard remedy for partial observability. We run it both alone and combined with each curriculum.
The ablation that isolates our key ingredient is a staged curriculum that visits exactly the same variants for exactly the same number of steps as the integrated one, differing only in hard block boundaries versus mixing. A second ablation removes the easier variants but keeps sensing cost ramping from zero to target. This separates "mixing tasks" from "gradually raising the price of information."
Integrated training beats the field only if it meets all of the following. Its held-out profit must exceed direct PPO and the asymmetric critic by more than the seed-to-seed spread across at least ten seeds. Its nitrate leaching must be no higher. It must beat the staged ablation, which shows that mixing, not merely exposure to easy variants, is responsible.
Why not just…?
Why not just train directly on the costly-sensing target with more samples? gym-DSSAT already needs on the order of $10^6$ cultivation cycles at full observability. Direct training on a harder partially observed task may simply never discover that buying a soil test pays off, because the reward for doing so arrives seasons later. Our equal-budget comparison measures this. We also run direct PPO at twice and four times the budget. If the curriculum's advantage disappears at larger budgets, we will report integrated training as a sample-efficiency gain only, not a gain in final performance.
Why not just use a standard staged curriculum from short to long horizons and from full to partial observation? This is the natural reading of the CyclesGym one-year-to-five-year result, and it is the control that tests the fragmentation hypothesis directly. The staged ablation uses identical variants and step counts. If it matches integrated training, the education-derived claim is wrong for agents, and we will say so. The nitrate probe and the performance dip at each stage switch will show whether staged agents fail for the predicted reason.
Why not just give the critic privileged soil information and skip the curriculum? This is the most serious threat. Asymmetric actor-critic training addresses the same information gap with no curriculum machinery. We treat it as a primary competitor and also cross it with each curriculum. If the privileged critic alone matches integrated training, our contribution shrinks to the costly-sensing benchmark and the diagnostic analysis. That is still useful, but it is a different paper, and we will present it that way.
Why not just use the expert split-fertilization rule, which needs no sensing at all? Split schedules do capture much of the value, as the orchard trial shows. But Analyzing Reinforcement Learning Algorithms for Nitrogen Fertilizer Management in Simulated Crop Growth found that fixed and reactive rules misallocate nitrogen in unusual weather years. Our evaluation reports profit separately for dry, wet, and typical held-out years, so we can see whether learned policies win only in unusual years or across the board.
Proposed Experiments
The main experiment runs every condition in CyclesGym with ten seeds. We train on 1980–2005 weather and test on 2006–2015 at both Pennsylvania sites. We report five-year profit net of soil-test costs, yield, total nitrogen applied, nitrate leaching, soil tests purchased, and environment steps needed to reach 90% of final performance.
The replication runs the same conditions in gym-DSSAT on the Florida rainfed maize setup with stochastic weather. This is the setting where the published full-observation agent cut leaching to 33 kg/ha and where partial observability was reported to hurt most. A secondary check in CropGym uses its fixed test years, including the dry 2014 season. It confirms the ranking holds under a different crop model and asks whether costly-sensing agents learn to withhold fertilizer that year the way the published Dueling DDQN did.
We will also sweep soil-test cost across three levels. The claim predicts that the advantage of integrated training grows as information becomes more expensive, because that is when a policy that learned to rely on free nitrate has the most to lose.
The decisive checks are these. If the staged ablation matches integrated training, the fragmentation hypothesis is refuted for agents. If the asymmetric critic alone matches integrated training, the curriculum is unnecessary. If integrated training wins on profit but its nitrate probe is no better than the staged agent's, the proposed mechanism is wrong even if the method works. We will report whichever of these outcomes occurs.
References
- Learning Long-Term Crop Management Strategies with CyclesGym
- Optimizing Nitrogen Management with Deep Reinforcement Learning and Crop Simulations
- gym-DSSAT: a crop model turned into a Reinforcement Learning environment
- Analyzing Reinforcement Learning Algorithms for Nitrogen Fertilizer Management in Simulated Crop Growth
- Combined Controlled-Release and Common Fertilizer Application Increases Apple Productivity by Optimizing Soil Nutrient and Microbial Communities
- An Enhanced Faster R-CNN for High-Throughput Winter Wheat Spike Monitoring to Improved Yield Prediction and Water Use Efficiency
- Towards connecting discrete mathematics and software engineering
- AI Education Across the Curriculum: Design and Pilot Study of a Cross-Disciplinary Module Set
- Architecture-AI Curriculum Integration: Theoretical Frameworks and Pedagogical Innovations