Wheat Yield Prediction Based on Parallel CNN-LSTM-Attention with Transfer Learning Model

Caixia Song, Tommy Liu, Wanshan Ning, Tong Xu, Shuhui Song, Zifei Li, Songtao Ouyang, Xing Song, T. Yong-Jin Han, Zichen Zhang, Tianyu Chen, Jinbao XieAgriculture 2025 · DOI

TL;DR

This paper introduces a novel Transfer-learning-based Parallel CNN-LSTM-Attention (TPCLA) model to improve wheat yield prediction, especially in data-scarce regions. By combining spatial and temporal feature extraction with cross-regional knowledge transfer and incorporating socio-economic factors alongside climate data, TPCLA achieves state-of-the-art accuracy, outperforming traditional models.

What Problem Does This Paper Address?

Accurate and timely wheat yield prediction is crucial for global food security and effective agricultural policy. However, two major challenges hinder current prediction efforts:

  1. Data Scarcity: Many regions lack long-term, high-quality agricultural time-series data, making it difficult to train robust predictive models.
  2. Factor Complexity: Wheat yields are influenced by a complex interplay of meteorological factors (temperature, precipitation) and increasingly important socio-economic factors (mechanization, policy, market trends). Traditional models often overlook these indirect socio-economic drivers, which modern agricultural practices use to buffer climate variability.

This study aims to bridge this gap by developing a model that can effectively learn from limited data and comprehensively consider both direct and indirect yield-influencing factors.

Methods

The core of this research is the Transfer-learning-based Parallel CNN-LSTM-Attention (TPCLA) model. Here's a breakdown of its key components:

  • Parallel CNN-LSTM Architecture: This is a novel approach that processes spatial and temporal features simultaneously.
  • CNN Branch: Uses 1D convolutional neural networks to extract localized spatial patterns from features like cultivated land area or disaster-affected regions.
  • LSTM Branch: Employs stacked Long Short-Term Memory (LSTM) networks to capture long-term temporal dependencies and seasonal dynamics in factors like rainfall and temperature.
  • Feature Fusion: The spatial and temporal features extracted by CNN and LSTM are concatenated to create a unified spatiotemporal representation.
  • Attention Mechanism: Integrated after feature fusion, this mechanism assigns adaptive weights to critical time steps within the fused representation. This helps the model focus on the most influential temporal features, enhancing interpretability and predictive power.
  • Cross-Regional Transfer Learning: To combat data scarcity, the model utilizes a transfer learning strategy.
  • Pre-training: The model is first trained on wheat datasets from ecologically similar regions to learn general, invariant yield-influencing patterns.
  • Fine-tuning: The pre-trained model is then fine-tuned on the target region's data (Shandong, in this case) to adapt to local characteristics. This "learn-and-adapt" strategy is crucial for small-sample scenarios.
  • Inclusion of Socio-Economic Factors: Unlike many previous studies, TPCLA explicitly incorporates indirect socio-economic indicators such as agricultural mechanization level, total agricultural output value, and disaster-affected area, alongside traditional meteorological variables.

Datasets

The study utilizes multivariate wheat time-series data sourced from government statistical yearbooks of multiple provinces in China.

  • Data Period: 1993 to 2024.
  • Features: The dataset includes eight features: rainfall, average temperature, total cultivated land area, total grain output, gross output value of agriculture, forestry, animal husbandry, and fishery, total agricultural output value, total power of agricultural machinery, and disaster-affected area.
  • Target Variable: Wheat yield per unit area.
  • Data Split:
    • Training: 1993-2012
    • Verification: 2013-2016
    • Testing: 2017-2024

Key Insights and Ideas

  • Socio-economic Factors Dominate: Surprisingly, the study found that indirect socio-economic indicators (like agricultural machinery power) often have a greater influence on wheat yields than traditional meteorological variables. This highlights how modern agriculture's ability to buffer climate impacts is increasingly shaping yield outcomes.
  • Transfer Learning is Key for Small Data: The cross-regional transfer learning approach significantly improves prediction accuracy and robustness, especially when dealing with limited historical data in a specific region. It allows the model to learn general patterns before specializing.
  • Parallel Processing Enhances Spatiotemporal Understanding: Processing spatial and temporal features in parallel, rather than sequentially, preserves the original temporal structure of the data and avoids distortions, leading to a more comprehensive representation.
  • Attention Improves Focus and Interpretability: The attention mechanism effectively identifies and weights critical temporal features, making the model more interpretable and allowing it to pinpoint key influencing moments.
  • Model as a Diagnostic Tool: The observed systematic prediction biases (over/underestimation in certain years) suggest the influence of uncaptured external factors (e.g., extreme weather, policy changes). This indicates the model can serve not just for prediction but also for identifying potential drivers of yield deviations.

Key Results

  • Superior Performance: The proposed TPCLA model achieved the best performance across all evaluation metrics on the test dataset.
  • RMSE: 0.394
    • MAE: 0.326
    • R²: 0.904
  • Outperforms Benchmarks: TPCLA significantly outperformed baseline models including RNN, LSTM, LSTM-Attention, and PCLA. For instance, compared to the LSTM-Attention model, TPCLA showed an 18.36% reduction in RMSE and a 12.60% decrease in MAE.
  • Robustness: The model demonstrated strong stability and generalization capability, particularly in the later years of the test period (2021-2024).
  • Feature Importance: Total power of agricultural machinery was identified as the most influential positive factor, while disaster-affected area showed a negative correlation with wheat yields.

Limitations and Future Work

  • Unobserved External Factors: The study acknowledges that systematic prediction biases in certain years point to the influence of external drivers not included in the feature set (e.g., extreme climate events, policy shifts).
  • Trending Socio-economic Indicators: Future work should address the issue of inherently trending socio-economic indicators (like total agricultural output value) that might introduce spurious correlations.
  • Future Research Directions:
    • De-trending or economically normalizing long-term socio-economic indicators.
  • Employing causality-aware feature selection or structural causal models to disentangle genuine yield determinants.
  • Designing trend-robust architectures that separate short-term agronomic signals from long-term economic drift.
  • Further enhancing model interpretability and preventing biased correlations.

Summary updated

Abstract

Accurate wheat yield prediction is essential for ensuring food security and supporting governmental decision-making. However, the scarcity of long-term agricultural time-series data and the complex interplay between meteorological and socio-economic factors pose significant challenges. To address these issues, this study proposes a Transfer-Learning-Based Parallel CNN–LSTM–Attention (TPCLA) model for wheat yield forecasting. A cross-regional transfer learning strategy is employed to mitigate data scarcity by leveraging temporal patterns learned from regions with similar ecological characteristics. The proposed parallel architecture integrates one-dimensional convolutional neural networks and long short-term memory networks to jointly extract spatial and temporal features, while an attention mechanism is incorporated to highlight key influencing factors and enhance feature interpretability. Unlike conventional studies that primarily focus on climatic variables, this work considers both direct factors (e.g., average temperature and precipitation) and indirect socio-economic factors (e.g., agricultural mechanization level, total agricultural output value, grain production scale, cultivated land area, and disaster-affected area). Experimental results on multivariate wheat data from 1993 to 2024 demonstrate that several indirect indicators exert a more substantial influence on yield than traditional meteorological variables—reflecting the increasing ability of modern agricultural practices to buffer climatic variability. The proposed TPCLA model achieves an RMSE of 0.394, MAE of 0.326, and an R2 of 0.904, outperforming multiple benchmark models and confirming its robustness and predictive superiority under small-sample conditions. The findings not only validate the effectiveness of integrating indirect yield-influencing factors but also provide new insights for agricultural policy formulation and climate resilience strategies.
BibTeX
@inproceedings{song2025wheat,
  author = {Caixia Song and Tommy Liu and Wanshan Ning and Tong Xu and Shuhui Song and Zifei Li and Songtao Ouyang and Xing Song and T. Yong-Jin Han and Zichen Zhang and Tianyu Chen and Jinbao Xie},
  title = {{Wheat Yield Prediction Based on Parallel CNN-LSTM-Attention with Transfer Learning Model}},
  booktitle = {Agriculture},
  year = {2025},
  doi = {10.3390/agriculture15232519},
}
Figure