TL;DR
This paper introduces PC4C_CAPSI, a novel RGBD dataset capturing the growth of hydroponically grown capsicum plants in a commercial greenhouse over 20 weeks. This dataset addresses the critical need for diverse, real-world data to advance computer vision for automated crop monitoring, particularly for plant height estimation and 3D reconstruction in complex protected horticulture environments.
What Problem Does This Paper Address?
The global challenge of feeding a growing population while minimizing environmental impact has led to increased adoption of protected horticulture. However, monitoring crops in these sophisticated, often restricted, commercial facilities is largely manual and labor-intensive. Existing computer vision research for crop monitoring is hampered by a lack of comprehensive, real-world datasets, especially for vertically supported crops in complex greenhouse environments. This scarcity limits the development and validation of robust automated monitoring systems. This paper aims to bridge this gap by providing a rich RGBD dataset that captures the full growth cycle of capsicum plants under varying light conditions, enabling research into plant height estimation, organ identification, and 3D reconstruction.
Methods
The core of this work is the creation and release of the PC4C_CAPSI dataset. Data was collected weekly over 20 weeks using an Intel Realsense D415 depth camera mounted on a mobile platform. This setup allowed for a top-angled view of vertically supported capsicum plants in a commercial glasshouse. The data collection covered approximately 80% of the crop's growing season and was conducted under three different light conditions (diffused light, Smart Glass film, and LLEAF film).
Key methodological aspects include:
- RGBD Data Capture: The Realsense D415 camera captures both color (RGB) and depth information, providing rich 3D scene understanding.
- Mobile Platform: A trolley-mounted platform enabled consistent movement along the hydroponic gutters, facilitating weekly data collection.
- Data Organization: Data is organized into Rosbag (.bag) files containing RGBD streams and associated metadata, and JSON (.json) files containing metadata and ground truth.
- Ground Truth: Manually measured plant heights for ten selected plants per gutter are provided as ground truth for plant height estimation tasks.
- Depth Calibration: A calibration error in the depth camera was identified and corrected through an indoor experiment and linear regression, with the methodology and data detailed in the paper.
- Image Extraction and Preprocessing: A Python script using the Pyrealsense library was developed for image extraction from Rosbag files, including post-processing filters like decimation, temporal, and spatial filters.
Datasets
The paper introduces the PC4C_CAPSI dataset. This is a newly collected dataset comprising:
- RGBD streams: Captured weekly over 20 weeks.
- Metadata: Including camera configurations and light conditions.
- Ground Truth: Manually measured plant heights for selected plants.
This dataset differs from existing ones by focusing on vertically supported crops in a commercial-scale, high-tech glasshouse, capturing the full growth cycle, and providing both RGB and depth information. It is made available through the ResearchDirect repository.
Key Insights and Ideas
- The critical need for real-world, complex data: The paper highlights that while protected horticulture is growing, the data available for training computer vision models is scarce and often limited to controlled lab settings or less complex crops. This dataset directly addresses this by providing data from a commercial, operational environment.
- RGBD data's power for 3D understanding: The use of RGBD cameras is crucial for capturing not just visual appearance but also spatial information, enabling 3D reconstruction and more accurate measurements like plant height, which is difficult with RGB alone.
- Challenges in automated crop monitoring: The paper implicitly demonstrates the complexities involved, such as varying light conditions, plant growth obscuring lower parts, and the need for robust plant identification and tracking methods.
- Depth calibration is essential: The authors proactively identified and addressed a depth calibration error, underscoring the importance of meticulous data quality control for reliable computer vision applications in agriculture.
Key Results
- The PC4C_CAPSI dataset contains RGBD streams and metadata for capsicum plants over an average of 20 weeks.
- Data was collected from three different light conditions within the glasshouse.
- Manually measured plant heights for ten selected plants per gutter are provided as ground truth.
- The dataset has been successfully demonstrated for plant height estimation in a prior study, indicating its immediate utility.
- The paper details a depth calibration correction methodology and provides the data collected for this experiment.
Limitations and Future Work
Limitations: * Top-angled view: The data primarily captures the top third of the plants due to the camera's top-angled perspective, limiting visibility of the middle and lower canopy for feature extraction. * Depth noise: Depth images exhibit noise that is proportional to the distance from the camera, making the lower canopy noisier. * Manual platform movement: The manual movement of the imaging platform means speed may not be constant, potentially introducing variability. * Natural light variations: Images are captured under natural light, leading to potential noise from bright spots and shading. * Ground truth availability: Ground truth is primarily available for plant height, and there are missing values for three weeks. * Data gaps for LLEAF film: The last four time points for the LLEAF film room are unavailable due to hardware limitations.
Future Work: The authors suggest that this dataset can support research in:
- Plant height estimation.
- Plant organ identification and segmentation.
- Object segmentation.
- Organ measurements.
- 3D reconstruction and data processing.
- Depth noise reduction and hole filling.
- Developing robust object tracking algorithms for plants.
Summary updated