A Bayesian Network approach to County-Level Corn Yield Prediction using historical data and expert knowledge

Vikas Chawla, Hsiang Sing Naik, Adedotun Akintayo, Dermot Hayes, Patrick S. Schnable, Baskar Ganapathysubramanian, Soumik SarkararXiv 2016 · arXiv

Introduction

Predicting crop yield is a high-stakes endeavor for futures markets, insurance providers, and policy makers. Traditionally, researchers have relied on either "White Box" mechanistic models, which are scientifically grounded but computationally heavy and hard to calibrate, or "Black Box" statistical models, which are powerful but often lack interpretability and ignore underlying causal relationships. This paper introduces a "gray box" approach that bridges this gap by combining 64 years of historical data with expert agronomic knowledge through Bayesian Networks (BN).

Methodology

The core of this approach is the development of a Directed Acyclic Graph (DAG) that represents the probabilistic relationships between various environmental factors and corn yield. The process follows three main stages:

  1. Variable Selection & Preprocessing: The researchers curated direct variables (rainfall, temperature) and derived agronomic variables such as Growing Degree Days (GDD), the Palmer Drought Severity Index (PDSI), and the Corn Suitability Rating (CSR2).
  2. Incorporating Expert Knowledge: To ensure the model respects biological and meteorological laws, experts "pruned" the graph. This involved forbidding impossible connections (e.g., future events affecting the past) and enforcing known strong links, such as the relationship between rainfall and yield.
  3. Learning & Inference: A Bayesian search algorithm identifies the network structure that best explains the training data. As a probabilistic model, it can provide an "expected yield" even when mid-season data is missing.
Methodological workflow for the Bayesian Network yield prediction model, from data collection to inference.

Data and Experimental Setup

The study focused on the state of Iowa, utilizing an extensive historical dataset spanning 1948 to 2012. Data was aggregated for all 99 Iowa counties from the National Agricultural Statistics Service (NASS), the Global Historical Climatology Network (GHCN), and the Soil Survey Geographic (SSURGO) database. The model was trained on data from 2005–2009 and validated against actual 2010 yields.

Temporal Tiering and Causal Structure

A key innovation in this model is the organization of variables into "temporal tiers." This structure ensures the model respects the arrow of time, preventing the "black box" issue where a model might erroneously correlate high yields with a variable that has not yet occurred. This tiering allows the model to function effectively with "gappy" data; for instance, it can provide a baseline prediction in June using available evidence and refine that prediction as later "bins" (like August weather data) are filled.

Directed Acyclic Graph (DAG) illustrating the organization of variables into temporal tiers to ensure causal consistency.

Key Results and Discussion

The Bayesian Network model demonstrated high predictive power and practical utility:

  • High Accuracy: The model achieved an accuracy of 80% or higher for 70 out of 99 Iowa counties.
  • Impact of Evidence: In a case study of Polk County, an early-season prediction (May–June) had a 30% error rate. However, once full-season data and soil ratings (CSR2) were incorporated, the error dropped significantly to just 2%.
  • What-If Simulations: The causal mapping allows users to perform simulations, such as observing how the probability distribution for yield shifts if a severe drought occurs in July.
  • Trend Validation: The learned structure successfully corroborated known agronomic trends, such as the negative impact of extreme PDSI values (both extreme drought and extreme wetness) on final yield.

Limitations and Future Work

While the current results are promising, the authors aim to expand the model's variable set to include seed hybrids and management practices like fertilizer use. Additionally, increasing temporal resolution from monthly bins to daily or weekly scales could help capture the impact of short-term extreme weather events. The ultimate goal is to refine the model to consistently predict yield within a 5 bushels per acre margin for every county while adapting to long-term climate change patterns.

Summary updated

Abstract

Crop yield forecasting is the methodology of predicting crop yields prior to harvest. The availability of accurate yield prediction frameworks have enormous implications from multiple standpoints, including impact on the crop commodity futures markets, formulation of agricultural policy, as well as crop insurance rating. The focus of this work is to construct a corn yield predictor at the county scale. Corn yield (forecasting) depends on a complex, interconnected set of variables that include economic, agricultural, management and meteorological factors. Conventional forecasting is either knowledge-based computer programs (that simulate plant-weather-soil-management interactions) coupled with targeted surveys or statistical model based. The former is limited by the need for painstaking calibration, while the latter is limited to univariate analysis or similar simplifying assumptions that fail to capture the complex interdependencies affecting yield. In this paper, we propose a data-driven approach that is "gray box" i.e. that seamlessly utilizes expert knowledge in constructing a statistical network model for corn yield forecasting. Our multivariate gray box model is developed on Bayesian network analysis to build a Directed Acyclic Graph (DAG) between predictors and yield. Starting from a complete graph connecting various carefully chosen variables and yield, expert knowledge is used to prune or strengthen edges connecting variables. Subsequently the structure (connectivity and edge weights) of the DAG that maximizes the likelihood of observing the training data is identified via optimization. We curated an extensive set of historical data (1948-2012) for each of the 99 counties in Iowa as data to train the model.
BibTeX
@misc{chawla2016bayesian,
  author = {Vikas Chawla and Hsiang Sing Naik and Adedotun Akintayo and Dermot Hayes and Patrick S. Schnable and Baskar Ganapathysubramanian and Soumik Sarkar},
  title = {{A Bayesian Network approach to County-Level Corn Yield Prediction using historical data and expert knowledge}},
  year = {2016},
  eprint = {1608.05127},
  archivePrefix = {arXiv},
  url = {https://arxiv.org/abs/1608.05127},
}
Figure