Research on the Relative Position Detection Method between Orchard Robots and Fruit Tree Rows

Baoxing Gu, Qin Liu, Yi Gao, Guangzhao Tian, Baohua Zhang, Haiqing Wang, He Li

TL;DR

This paper introduces an improved YOLOv4-based computer vision system for orchard robots to accurately detect fruit tree rows. By integrating an efficient channel attention mechanism (ECA) into the YOLOv4 architecture, the system achieves higher detection accuracy and significantly reduces errors in estimating the robot's heading angle and lateral deviation, crucial for autonomous navigation.

What Problem Does This Paper Address?

Autonomous navigation for agricultural robots in orchards is essential for improving efficiency and reducing labor. A key challenge is accurately determining the robot's relative position to the fruit tree rows, especially when traditional GPS signals are obstructed by dense canopies. Existing methods often struggle with low accuracy in estimating these positional parameters, hindering reliable autonomous navigation. This paper aims to bridge this gap by proposing a robust machine vision approach to precisely detect fruit tree trunks and calculate the robot's pose relative to the tree rows.

Methods

The core of this research lies in an enhanced object detection model and a novel pose estimation pipeline.

  • Improved Fruit Tree Trunk Detection: The authors leverage the YOLOv4 object detection framework, a state-of-the-art single-stage detector known for its speed and accuracy. To boost performance in the complex orchard environment, they introduce an ECA (Efficient Channel Attention) module into the PANet (Path Aggregation Network) module of YOLOv4. This ECA module helps the network focus on more informative channel features, leading to better detection of fruit tree trunks. The resulting model is named ECA5-YOLOv4.
  • Binocular Vision for 3D Positioning: A binocular camera system is employed to capture stereo images of the orchard.
  • Camera Coordinates: The detected fruit tree trunks from both cameras are matched. Using the principle of binocular triangulation, the 3D camera coordinates of the tree trunks are calculated.
  • Ground Projection: These camera coordinates are then transformed into ground projection coordinates.
  • Navigation Path Calculation: The midpoints of the projected coordinates of tree trunks on opposite sides of the robot are computed. A straight line is then fitted to these midpoints using the least squares method to define the navigation path.
  • Pose Parameter Estimation: Finally, the heading angle and lateral deviation of the robot are calculated based on the fitted navigation path and the robot's coordinate system.

Datasets

  • Training and Validation: The study utilized a dataset of 700 fruit tree trunk images collected from the College of Engineering, Nanjing Agricultural University. Data augmentation techniques (adding noise, flipping) expanded this to 1500 images, with 1200 for training and 300 for validation. The "tree" category was used for labeling.

Key Insights and Ideas

  • Attention Mechanisms are Crucial: The integration of the ECA module significantly improved the detection accuracy of fruit tree trunks. This highlights the importance of attention mechanisms in computer vision tasks for complex, natural environments where subtle features matter.
  • Binocular Vision for Robust Relative Pose: Relying on binocular vision for relative pose estimation offers a compelling alternative to GPS, especially in environments prone to signal obstruction. The triangulation and coordinate transformation pipeline effectively translates 2D image detections into actionable 3D positional information.
  • Least Squares for Path Fitting: Using the least squares method to fit a straight line to the projected tree trunk midpoints provides a stable and mathematically sound way to define the robot's intended path, even with noisy detections.
  • Low Error is Achievable: The experimental results demonstrate that this combined approach can achieve very low errors in heading angle and lateral deviation, making it practical for real-world autonomous navigation.

Key Results

  • Improved Detection Accuracy: The ECA5-YOLOv4 model achieved a 5.92% higher precision rate and a 7.91% higher recall rate compared to the original YOLOv4 model.
  • Accurate Pose Estimation: The proposed method achieved mean errors of 0.57° for heading angle and 0.02 m for lateral deviation. These are significantly lower than those achieved by other tested models (original YOLOv4, SENet5-YOLOv4, CBAM5-YOLOv4).
  • Competitive Frame Rate: While not the absolute fastest, the ECA5-YOLOv4 model maintained a competitive frame rate of 17.59 frames per second, which is sufficient for real-time navigation.

Limitations and Future Work

  • Limitations:
    • The study focused on detecting fruit tree trunks. The presence of dense foliage or branches might still pose challenges for accurate trunk identification.
  • The dataset size, while augmented, is relatively modest (1500 images).
  • The experiment was conducted in a specific orchard environment; generalization to vastly different orchard types or conditions might require further validation.
  • Future Work:
    • The authors suggest that their method provides a theoretical basis for autonomous visual navigation of orchard robots, implying future work could involve integrating this system into a full navigation and control framework.
  • Further research could explore more advanced attention mechanisms or object detection architectures to further enhance robustness in challenging visual conditions.
  • Expanding the dataset with more diverse orchard scenarios would be beneficial.

Summary updated

Abstract

The relative position of the orchard robot to the rows of fruit trees is an important parameter for achieving autonomous navigations. The current methods for estimating the position parameters between rows of orchard robots obtain low parameter accuracy, and to address this problem, this paper proposes a machine vision-based method for detecting the relative position of orchard robots and fruit tree rows. Firstly, the fruit tree trunk is identified based on the improved YOLOv4 model; secondly, the camera coordinates of the tree trunk are calculated from the principle of binocular camera triangulation, and the ground projection coordinates of the tree trunk are obtained through coordinate conversion; finally, the midpoints of the projection coordinates of different sides are combined and the navigation path is obtained by linear fitting with the least squares method, and the position parameters of the orchard robot are obtained through calculation. The experimental results show that the average accuracy and average recall of the improved YOLOv4 model for fruit tree trunk detection are 97.05% and 95.42%, respectively, which are 5.92 and 7.91 percentage points higher than those of the original YOLOv4 model. The average errors of heading angle and lateral deviation estimates obtained based on the method in this paper are 0.57° and 0.02 m. The method can accurately calculate heading angle and lateral deviation values at different positions between rows, and can provide a reference for autonomous visual navigation of orchard robots.
BibTeX
@misc{gu2023research,
  author = {Baoxing Gu and Qin Liu and Yi Gao and Guangzhao Tian and Baohua Zhang and Haiqing Wang and He Li},
  title = {{Research on the Relative Position Detection Method between Orchard Robots and Fruit Tree Rows}},
  year = {2023},
  doi = {10.3390/s23218807},
}
Figure