Improving Lives of Indebted Farmers Using Deep Learning: Predicting Agricultural Produce Prices Using Convolutional Neural Networks

Hangzhi Guo, Alexander Woodruff, Amulya YadavAAAI 2020 · DOI

Introduction: The Crisis of Agrarian Distress

In India, the agricultural sector is marred by a tragic phenomenon: over 300,000 farmer suicides since 1995. This "agrarian distress" is largely fueled by insurmountable debt, often triggered by extreme price volatility in agricultural markets. Small-scale farmers, lacking sophisticated forecasting tools, frequently sell their produce at a loss or face total crop loss when local prices crash. To address this, the authors propose PECAD (Price Estimation for Crops using the Application of Deep Learning), a novel architecture designed to provide accurate price predictions, empowering farmers to make informed decisions about when and where to sell.

Methodology: The PECAD Architecture

Existing price forecasting models often struggle with spatio-temporal dependencies and significant data sparsity. PECAD addresses these by adopting a "Wide and Deep" learning paradigm, modified for the complexities of agricultural data.

The PECAD architecture combining a TCN-infused Wide Network with a Deep Network.

The architecture consists of two primary components:

  1. TCN-Infused Wide Component: Unlike traditional wide models that use linear transformations, PECAD utilizes two Temporal Convolutional Networks (TCNs). These networks process historical price and volume sequences separately, capturing non-linear temporal patterns more effectively than standard recurrent models.
  2. Embedding-Based Deep Component: This component handles high-dimensional categorical features, such as specific Market and Crop IDs, by mapping them into low-dimensional embeddings. It also integrates spatial information (latitude and longitude) to account for regional price correlations.

To handle missing data—a common issue in government records—the researchers employed SoftImpute, a collaborative filtering technique that leverages geographic proximity to estimate missing values.

Dataset and Experimental Setup

The study utilizes an extensive dataset from Agmarknet.gov.in, covering 11 years (2008–2018) of daily data across approximately 1,352 Indian markets. The focus was on three volatile commodities: Brinjal, Tomato, and Chilli. Spatial data was augmented using the Google Maps API to provide precise coordinates for each market.

Key Insights and Results

The evaluation of PECAD revealed significant performance gains over both classical and standard deep learning baselines. Using the Coefficient of Variation as a metric, PECAD outperformed classical models like Random Forest by 12.5% and standard LSTM-based architectures by 25%.

Convergence rates and ablation results showing the importance of the 100-150 day window.

A critical insight emerged from the ablation studies (shown in the figure above). The model exhibited a sharp increase in error when data from 100 to 150 days in the past was masked. This window corresponds to the typical 3–4 month growing cycle for the studied crops, indicating that the model successfully learned the underlying biological and supply-chain realities of farming. Furthermore, the inclusion of supply volume as an independent feature proved vital, as gluts in supply are strong precursors to price drops.

Conclusion and Future Directions

PECAD demonstrates the potential of deep learning to mitigate economic risks for vulnerable populations. While the model is highly accurate, its real-world impact depends on successful deployment. Future work includes integrating weather data (rainfall and temperature) to further refine supply predictions and collaborating with NGOs to develop accessible, trust-based digital tools for farmers. By bridging the gap between advanced AI and grassroots farming, PECAD offers a path toward reducing debt-driven distress in rural India.

Summary updated

BibTeX
@inproceedings{guo2020improving,
  author = {Hangzhi Guo and Alexander Woodruff and Amulya Yadav},
  title = {{Improving Lives of Indebted Farmers Using Deep Learning: Predicting Agricultural Produce Prices Using Convolutional Neural Networks}},
  booktitle = {AAAI Conference on Artificial Intelligence},
  year = {2020},
  doi = {10.1609/aaai.v34i08.7039},
}

Loading related content...

Figure