Automatic Non-Destructive Growth Measurement of Leafy Vegetables Based on Kinect

Yang Hu, Le Wang, Lirong Xiang, Qian Wu, Huanyu JiangSensors 2018 · DOI

TL;DR

This paper presents an automated, low-cost system using a Kinect v2 sensor to perform non-destructive 3D growth measurements of leafy vegetables. By acquiring multi-view point clouds and employing a suite of processing algorithms, the system accurately estimates key growth parameters like height, leaf area, and volume, showing strong correlations with ground truth and biomass.

What Problem Does This Paper Address?

Accurate and non-destructive measurement of plant growth is crucial for agricultural research, phenotyping, and crop forecasting. Traditional manual methods are time-consuming and labor-intensive. While various 3D sensing technologies exist (stereo vision, structured light, ToF cameras), they often face challenges with accuracy, cost, complexity, or noise. Specifically, consumer-grade RGB-D sensors like Kinect v2 offer a compelling balance of affordability and robustness, but their inherent noise and the complexity of reconstructing delicate plant structures present significant hurdles for precise growth parameter estimation. This paper aims to overcome these challenges by developing an integrated system that leverages Kinect v2 for detailed 3D plant analysis.

Methods

The core of the system involves several key steps:

  • Data Acquisition: A Kinect v2 sensor captures multi-view point clouds of a plant placed on a motorized turntable. The turntable rotates in 20° increments, capturing 18 views per plant. A white card is used to improve color balance in the captured images.
  • Point Cloud Processing:
    • Noise Reduction: A two-stage approach is used: Multi-View Interference Elimination (MIE) to address layered points caused by light refraction, followed by Moving Least Squares (MLS) to smooth remaining local noise on leaf surfaces.
  • Triangulation: The denoised point cloud is down-sampled, and then a triangular mesh is generated. This involves fitting triangles to points, filtering abnormal triangles, and fixing holes.
  • Boundary Smoothing: The mesh boundaries are smoothed using a local weighted-average approach to mitigate jagged edges caused by leaf boundary noise.
  • Segmentation:
    • Plant/Non-Plant Segmentation: This is achieved using the HSI color space, leveraging the distinct green color of plants. Quadric surfaces are fitted to manually segmented data to create robust segmentation models.
  • Pot Shape Feature Extraction: To isolate the "in-pot" plant parts for height measurement, the system models the pot as a truncated cone. By fitting circles at different heights, the cone's apex is determined, allowing for the segmentation of soil and the in-pot plant.
  • Growth Parameter Measurement:
    • Height: Both relative (pot bottom to plant top) and absolute heights (soil-to-plant top, and plant bottom-to-plant top) are calculated.
  • Leaf Area: Total leaf area is computed by summing the areas of all triangles in the mesh. Projected leaf area is determined by projecting the mesh onto the x-y plane and counting points within the projection.
  • Volume: Volume is measured using a tetrahedralization approach, where overlapped tetrahedrons are constructed from mesh vertices. A matrix-based method is then used to count points within these tetrahedrons to estimate volume.

Datasets

  • Training/Evaluation Data: 63 pots of lettuce plants at various growth stages were used for experiments.
  • Reference Data: For validation, measurements were taken using manual methods (ruler for height, photography for projected area, water displacement for volume) and destructive methods (fresh/dry weight for biomass).
  • Benchmark: While not a formal benchmark, the paper provides a comprehensive dataset of Kinect measurements correlated with ground truth and biomass, which could serve as a reference for future studies.

Key Insights and Ideas

  • Kinect v2's Potential: The study demonstrates that despite its inherent noise, the Kinect v2 sensor, when combined with robust processing algorithms, can yield surprisingly accurate 3D plant measurements.
  • Two-Stage Noise Reduction: The combination of MIE and MLS algorithms is crucial for effectively cleaning the noisy point clouds of delicate plant structures.
  • HSI for Segmentation: Utilizing the HSI color space proves effective for distinguishing plant from non-plant parts, even with variations in local color.
  • Pot Shape as a Feature: Modeling the pot as a truncated cone provides a reliable method for segmenting the "in-pot" plant, essential for accurate height measurements.
  • Power Law Relationships: Total leaf area and volume measurements exhibit power-law relationships with reference data and biomass, suggesting that these parameters scale non-linearly with plant growth.
  • Volume as a Biomass Proxy: Volume measurements show a stronger correlation with biomass (fresh and dry weight) than total leaf area, making it a more reliable indicator of plant mass.

Key Results

  • Height Measurement: Kinect measurements showed good linear relationships with reference data. Relative height had the highest fitting goodness (R² = 0.9914). Absolute height measured by the soil method (R² = 0.9855) was more accurate than the plant bottom method (R² = 0.9457).
  • Leaf Area: Projected leaf area showed a good linear relationship (R² = 0.9510). Total leaf area followed a power-law distribution (R² = 0.9460).
  • Volume: Volume also followed a power-law distribution with reference data (R² = 0.9673).
  • Biomass Correlation: Volume showed a good power-law relationship with fresh weight (R² = 0.9281), indicating its potential as a biomass predictor.
  • Accuracy: Mean Absolute Percentage Errors (MAPEs) for linear relationships were generally low, with relative height having the lowest MAPE (2.58%).
  • Robustness: The system demonstrated good robustness across different sizes and species of leafy vegetables.
  • Time Consumption: The total processing time scales with the number of points, following a power-law distribution, with MIE denoising and volume computation being the most time-consuming steps.

Limitations and Future Work

  • Limitations:
    • Time Consumption: The system's throughput is limited, especially for large plants with many points, making it less ideal for applications requiring high speed.
  • Lighting Sensitivity: Plant/non-plant segmentation is sensitive to color, requiring stable lighting conditions.
  • Pot Shape Constraint: The current segmentation method is limited to truncated-cone-shaped pots.
  • Manual Placement: Plants need to be manually placed on the turntable, limiting full automation.
  • Stem Measurement: Thin stems are not well-captured due to Kinect's resolution.
  • Non-Green Vegetables: The HSI segmentation model may not be suitable for non-green plants.
  • Future Work:
    • Develop self-adaptive parameter selection for algorithms.
  • Optimize algorithms or use multithreading/GPU acceleration to speed up processing.
  • Adapt the system for more plant species and test robustness.
  • Introduce additional sensors (e.g., top-view camera) for more comprehensive data.
  • Utilize the acquired 3D data for more advanced studies, such as stress and disease detection.

Summary updated

Abstract

Non-destructive plant growth measurement is essential for plant growth and health research. As a 3D sensor, Kinect v2 has huge potentials in agriculture applications, benefited from its low price and strong robustness. The paper proposes a Kinect-based automatic system for non-destructive growth measurement of leafy vegetables. The system used a turntable to acquire multi-view point clouds of the measured plant. Then a series of suitable algorithms were applied to obtain a fine 3D reconstruction for the plant, while measuring the key growth parameters including relative/absolute height, total/projected leaf area and volume. In experiment, 63 pots of lettuce in different growth stages were measured. The result shows that the Kinect-measured height and projected area have fine linear relationship with reference measurements. While the measured total area and volume both follow power law distributions with reference data. All these data have shown good fitting goodness (<i>R</i>² = 0.9457-0.9914). In the study of biomass correlations, the Kinect-measured volume was found to have a good power law relationship (<i>R</i>² = 0.9281) with fresh weight. In addition, the system practicality was validated by performance and robustness analysis.
BibTeX
@inproceedings{hu2018automatic,
  author = {Yang Hu and Le Wang and Lirong Xiang and Qian Wu and Huanyu Jiang},
  title = {{Automatic Non-Destructive Growth Measurement of Leafy Vegetables Based on Kinect}},
  booktitle = {Sensors},
  year = {2018},
  doi = {10.3390/s18030806},
}
Figure