Crop Lodging Prediction from UAV-Acquired Images of Wheat and Canola\n using a DCNN Augmented with Handcrafted Texture Features

Sara Mardanisamani, Farhad Maleki, Kassani Sara Hosseinzadeh, Sajith Rajapaksa, Duddu Hema, Menglu Wang, Shirtliffe Steve, Seungbum Ryu, Anique Josuttes, Ti Zhang, Vail Sally, Pozniak Curtis, Parkin Isobel, Stavness Ian, Eramian Mark

Abstract

Lodging, the permanent bending over of food crops, leads to poor plant growth and development. Consequently, lodging results in reduced crop quality, lowers crop yield, and makes harvesting difficult. Plant breeders routinely evaluate several thousand breeding lines, and therefore, automatic lodging detection and prediction is of great value aid in selection. In this paper, we propose a deep convolutional neural network (DCNN) architecture for lodging classification using five spectral channel orthomosaic images from canola and wheat breeding trials. Also, using transfer learning, we trained 10 lodging detection models using well-established deep convolutional neural network architectures. Our proposed model outperforms the state-of-the-art lodging detection methods in the literature that use only handcrafted features. In comparison to 10 DCNN lodging detection models, our proposed model achieves comparable results while having a substantially lower number of parameters. This makes the proposed model suitable for applications such as real-time classification using inexpensive hardware for high-throughput phenotyping pipelines.

Introduction

In modern agriculture, "lodging"—the permanent displacement of crop stems from their vertical position—represents a significant challenge. Triggered by adverse weather conditions like heavy winds and rain, or inherent structural weaknesses in the plant, lodging can reduce yields by up to 50%, degrade grain quality, and complicate the mechanical harvesting process. For plant breeders, identifying lodging-resistant varieties is a priority, but the traditional method of manual inspection is labor-intensive, subjective, and difficult to scale across thousands of experimental plots.

UAV-acquired orthomosaic of breeding trials showing individual plots for wheat and canola.

While Unmanned Aerial Vehicles (UAVs) provide a high-throughput alternative for field monitoring, the automated analysis of these images remains difficult. Previous attempts have relied either on handcrafted features, which lack robustness, or massive Deep Convolutional Neural Networks (DCNNs), which require significant computational resources. This paper introduces LodgedNet, a lightweight hybrid model designed to bridge this gap by combining the strengths of deep learning with classical texture analysis.

Methodology: The LodgedNet Architecture

The core innovation of LodgedNet is its "augmented" architecture. Rather than relying solely on a deep stack of convolutional layers to learn features, the model explicitly incorporates domain-specific texture information.

1. CNN Backbone

The model utilizes a relatively shallow 7-layer DCNN. This backbone is responsible for automatically learning hierarchical spatial features from the input images. By keeping the network shallow, the researchers significantly reduced the parameter count compared to standard architectures like VGG or ResNet.

2. Handcrafted Texture Module

Simultaneously, the system extracts two types of classical texture features:

  • Local Binary Patterns (LBP): These capture local spatial structures and are highly effective at describing micro-textures.
  • Gray-Level Co-occurrence Matrix (GLCM): These features quantify the spatial relationship between pixels, helping the model identify the "directionality" and contrast of textures—critical indicators of lodged crops which often exhibit a distinct lean.
Visual comparison of crop textures: (a) non-lodged wheat, (b) lodged wheat, (c) non-lodged canola, and (d) lodged canola.

3. Feature Fusion

The handcrafted features are concatenated with the high-level features learned by the CNN. This fused feature vector is then passed through fully connected layers for final classification. This approach ensures that the model benefits from both the automated feature discovery of deep learning and the targeted textural insights of classical computer vision.

Experimental Setup and Datasets

The study utilized high-resolution imagery from breeding trials in Saskatoon, Canada. Data was collected using a Draganfly X4P quadcopter equipped with a MicaSense RedEdge camera, capturing five spectral channels: Red, Green, Blue, Near-Infrared (NIR), and Red-edge.

The dataset comprised:

  • Wheat: 465 plots across two large orthomosaic images.
  • Canola: 1,638 plots across seven orthomosaic images.

Ground truth labels were provided by professional agronomists. To ensure model robustness, the researchers employed data augmentation techniques, including random flips and rotations, to expand the training set.

Key Results and Discussion

LodgedNet demonstrated exceptional performance across both crop types, achieving an accuracy of 97.7% for wheat and 99.06% for canola.

Efficiency and Real-Time Potential

One of the most striking findings is the model's efficiency. LodgedNet contains only 332,306 parameters. In comparison, VGG19—a common benchmark—contains over 143 million parameters. This makes LodgedNet approximately 432 times smaller while maintaining comparable accuracy. With an inference time of roughly 3ms per plot, the model is ideally suited for deployment on low-power, inexpensive hardware within high-throughput phenotyping pipelines.

The Value of Spectral Data

The researchers also explored the impact of different spectral bands. While the model performed well with standard RGB data, the inclusion of NIR and Red-edge channels provided additional biological context that can be vital for other agricultural assessments, such as disease detection or nutrient stress analysis.

Conclusion and Future Work

LodgedNet proves that "deeper" is not always better. By integrating handcrafted texture features, the researchers created a model that is both highly accurate and computationally efficient. Future work will focus on optimizing the handcrafted feature extraction process, which currently acts as a minor CPU bottleneck. The team also plans to investigate the inclusion of Gabor filters to better capture oriented textures and test the architecture's generalizability to other major crops like rice and maize.

Summary updated

Abstract

Lodging, the permanent bending over of food crops, leads to poor plant growth\nand development. Consequently, lodging results in reduced crop quality, lowers\ncrop yield, and makes harvesting difficult. Plant breeders routinely evaluate\nseveral thousand breeding lines, and therefore, automatic lodging detection and\nprediction is of great value aid in selection. In this paper, we propose a deep\nconvolutional neural network (DCNN) architecture for lodging classification\nusing five spectral channel orthomosaic images from canola and wheat breeding\ntrials. Also, using transfer learning, we trained 10 lodging detection models\nusing well-established deep convolutional neural network architectures. Our\nproposed model outperforms the state-of-the-art lodging detection methods in\nthe literature that use only handcrafted features. In comparison to 10 DCNN\nlodging detection models, our proposed model achieves comparable results while\nhaving a substantially lower number of parameters. This makes the proposed\nmodel suitable for applications such as real-time classification using\ninexpensive hardware for high-throughput phenotyping pipelines. The GitHub\nrepository at https://github.com/FarhadMaleki/LodgedNet contains code and\nmodels.\n
BibTeX
@inproceedings{mardanisamani2019crop,
  author = {Sara Mardanisamani and Farhad Maleki and Kassani Sara Hosseinzadeh and Sajith Rajapaksa and Duddu Hema and Menglu Wang and Shirtliffe Steve and Seungbum Ryu and Anique Josuttes and Ti Zhang and Vail Sally and Pozniak Curtis and Parkin Isobel and Stavness Ian and Eramian Mark},
  title = {{Crop Lodging Prediction from UAV-Acquired Images of Wheat and Canola\textbackslash\{\}n using a DCNN Augmented with Handcrafted Texture Features}},
  booktitle = {Workshops @ CVPR},
  year = {2019},
  doi = {10.1109/cvprw.2019.00322},
}
Figure