Adaptive Symmetry Self-Matching for 3D Point Cloud Completion of Occluded Tomato Fruits in Complex Canopy Environments

Wen-Qin Wang, Chengda Lin, Haiyu Shui, Ke Zhang, Ruifang ZhaiPlants 2025 · DOI

TL;DR

This paper introduces an "Adaptive Symmetry Self-Matching" (ASSM) algorithm to reconstruct 3D models of occluded tomato fruits, overcoming limitations of traditional methods that struggle with missing data. By dynamically adjusting symmetry planes and using a multi-orthogonal system, ASSM significantly improves accuracy in complex agricultural environments, paving the way for more precise plant phenotyping.

What Problem Does This Paper Address?

Precise 3D reconstruction of fruits is crucial for smart agriculture, enabling accurate yield estimation and quality assessment. However, in dense greenhouse environments, leaves and stems often obscure fruits, leading to significant gaps (up to 60%) in collected 3D point cloud data. Traditional methods like geometric fitting (e.g., ellipsoids) are highly sensitive to this missing data, leading to inaccurate morphology estimation, especially for deformed or partially occluded fruits. Existing symmetry-based methods often rely on static symmetry assumptions, failing to adapt to dynamic growth or complex occlusion patterns. This paper tackles the challenge of accurately completing 3D fruit models when substantial portions of their point cloud data are missing due to complex canopy occlusion.

Methods

The core of this work is the Adaptive Symmetry Self-Matching (ASSM) algorithm. It's a dual-architecture approach combining "local detection-global morphological optimization."

Here's a breakdown of the key steps:

  • Initial Symmetry Plane Determination:
    • An ellipsoid model is fitted to the point cloud using RANSAC to find the fruit's geometric center.
  • Principal Component Analysis (PCA) is applied to the centered point cloud to find the eigenvector with maximum variance, which serves as the normal vector for the initial symmetry plane.
  • Adaptive Symmetry Plane Optimization:
    • The point cloud is divided into two halves based on the initial symmetry plane.
  • Edge contours are extracted from both the original and mirrored point clouds.
  • The Iterative Closest Point (ICP) algorithm is used to iteratively align these contours, dynamically adjusting the symmetry plane's orientation to minimize positional deviation. This step is crucial for adapting to local data defects.
  • Triple-Orthogonal Symmetry System Construction:
    • Once the primary symmetry plane is optimized, two additional symmetry planes are constructed, orthogonal to the first and intersecting the fruit's centroid. This creates a multi-directional symmetry constraint system.
  • Multi-Plane Mirroring and Completion:
    • Missing regions are mirrored across all three orthogonal symmetry planes to generate complementary point clouds.
  • ICP alignment is used again to ensure seamless integration of these mirrored regions with the original data.
  • Finally, voxel grid filtering is applied to downsample and smooth overlapping regions, resulting in a complete and continuous 3D fruit model.

The novelty lies in the adaptive and dynamic adjustment of symmetry planes and the construction of a multi-orthogonal symmetry system, moving beyond static, single-plane assumptions.

Datasets

The primary dataset used for training and evaluation consists of 150 tomato fruits collected from 42 tomato plants. These fruits exhibited occlusion rates ranging from 5% to 70%. The data was acquired using a FARO Focus S70 3D laser scanner under natural daylight conditions.

For cross-crop validation, 30 eggplant fruits were also used, with missing data rates from 10% to 70%.

Key Insights and Ideas

  • Symmetry is Key, But It Needs to Be Adaptive: The paper strongly emphasizes that fruits possess inherent symmetry, which is a powerful cue for reconstruction. However, simply assuming a fixed symmetry plane is insufficient. The adaptive nature of ASSM, which dynamically refines the symmetry plane based on real-time data characteristics, is a critical breakthrough.
  • Multi-Directional Symmetry for Complex Occlusions: Relying on a single symmetry plane is limiting. By creating a triple-orthogonal symmetry system, the algorithm can effectively "fill in" missing data from multiple angles, better handling complex, multi-directional occlusions common in dense plant canopies.
  • Overcoming Data Dependency: Traditional methods like ellipsoid fitting are brittle; they fail dramatically with significant data loss. ASSM's success demonstrates a path towards robust 3D reconstruction that is less dependent on the completeness of the initial point cloud.
  • Cross-Crop Applicability: The successful validation on eggplants suggests that the ASSM approach is not limited to tomatoes and can be generalized to other fruits with similar symmetrical properties, increasing its practical value.

Key Results

  • Improved Accuracy: ASSM achieved significantly higher R² values and lower RMSE and rRMSE compared to traditional ellipsoid fitting across low, medium, and high occlusion levels.
  • Under high occlusion, R² values for length, width, and height were 0.9914, 0.9880, and 0.9349, respectively.
  • RMSE was reduced by 23.51-56.10% compared to ellipsoid fitting.
  • Symmetry Plane Accuracy: The average angular error between the calculated and true symmetry planes (determined by peduncle direction) was 5.24° ± 2.23°, outperforming previous studies.
  • Robustness: Statistical analysis (Wilcoxon signed-rank test) confirmed the significant superiority of ASSM over ellipsoid fitting, especially under high occlusion.
  • Eggplant Validation: ASSM showed excellent performance on eggplant fruits, with R² values close to 1 and low RMSE/rRMSE, demonstrating cross-crop applicability.

Limitations and Future Work

  • Morphological Diversity: The paper acknowledges that extreme variations in fruit morphology, especially for irregular or asymmetrical shapes, might affect ASSM's accuracy. Natural shape variations can obscure clear symmetry features.
  • Future Directions:
    • Non-rigid Registration & Deformable Priors: To improve robustness for irregular fruits, incorporating non-rigid registration techniques and deformable shape priors is suggested.
  • Hybrid Deep Learning Integration: Exploring the integration of ASSM with deep learning models (e.g., PointNet++, PCN) is proposed. This could leverage the interpretability of geometric methods with the generalization capabilities of data-driven techniques, potentially improving performance on complex shapes and enabling more nuanced geometric analysis.

Summary updated

Abstract

As a globally important cash crop, the optimization of tomato yield and quality is strategically significant for food security and sustainable agricultural development. In order to address the problem of missing point cloud data on fruits in a facility agriculture environment due to complex canopy structure, leaf shading and limited collection viewpoints, the traditional geometric fitting method makes it difficult to restore the real morphology of fruits due to the dependence on data integrity. This study proposes an adaptive symmetry self-matching (ASSM) algorithm. It dynamically adjusts symmetry planes by detecting defect region characteristics in real time, implements point cloud completion under multi-symmetry constraints and constructs a triple-orthogonal symmetry plane system to adapt to multi-directional heterogeneous structures under complex occlusion. Experiments conducted on 150 tomato fruits with 5–70% occlusion rates demonstrate that ASSM achieved coefficient of determination (R2) values of 0.9914 (length), 0.9880 (width) and 0.9349 (height) under high occlusion, reducing the root mean square error (RMSE) by 23.51–56.10% compared with traditional ellipsoid fitting. Further validation on eggplant fruits confirmed the cross-crop adaptability of the method. The proposed ASSM method overcomes conventional techniques’ data integrity dependency, providing high-precision three-dimensional (3D) data for monitoring plant growth and enabling accurate phenotyping in smart agricultural systems.
BibTeX
@inproceedings{wang2025adaptive,
  author = {Wen-Qin Wang and Chengda Lin and Haiyu Shui and Ke Zhang and Ruifang Zhai},
  title = {{Adaptive Symmetry Self-Matching for 3D Point Cloud Completion of Occluded Tomato Fruits in Complex Canopy Environments}},
  booktitle = {Plants},
  year = {2025},
  doi = {10.3390/plants14132080},
}
Figure