Introduction
3D plant phenotyping—measuring physical traits like leaf area or stem height—is vital for modern agriculture. While Unmanned Ground Vehicles (UGVs) can capture stable images, they are expensive and limited by terrain. Drones (UAVs) are a cost-effective alternative, but they face a major technical hurdle: motion.
The Challenge of Motion in the Field
Standard 3D reconstruction algorithms, such as Neural Radiance Fields (NeRF) or 3D Gaussian Splatting (GS), assume the scene is perfectly still. However, in a field environment, wind and the drone's own "downwash" (the air pushed down by the rotors) cause leaves to flutter and bend. This violation of the "static scene" assumption results in blurry, noisy 3D models where fine details are lost. While dynamic reconstruction methods exist, they typically model motion over time. This research instead seeks a "canonical" reconstruction—a single, sharp, motion-compensated snapshot of the plant.
Methodology
The proposed solution consists of an autonomous capture system and an iterative reconstruction pipeline.
Autonomous Capture
Using a DJI Mini 3 Pro and a custom Android application, the drone flies autonomous circles around a plant. To solve the difficult problem of precise localization in a field, the system uses ArUco fiducial markers placed around the plant and an Extended Kalman Filter (EKF) to estimate the drone's pose.
Iterative Motion Compensation
The core innovation is an iterative pipeline that progressively warps input images toward a motion-free state. By using the initial (blurry) reconstruction as a spatial anchor and warping images toward this "average" position, the algorithm iteratively converges on a consistent, static geometry.
The process follows five key steps:
- Initial Training: Train a 3D model (e.g., 3D Gaussian Splatting) using the raw, moving images.
- Prediction: Render "predicted" images from the 3D model using the original camera viewpoints.
- Flow Estimation: Use RAFT (a deep learning model for optical flow) to calculate the displacement between the sharp (but moved) original images and the blurry (but "average" position) rendered images.
- Warping: Warp the original images using this flow to align them with the "canonical" 3D representation.
- Iteration: Repeat the process. Each iteration reduces scene motion, leading to a sharper 3D model.
Dataset and Results
The researchers are releasing a new dataset featuring 19 individual plants across various crops (Bean, Corn, Soy, and Sugar Beet) and 86 total scenes captured at different growth stages. The data includes camera parameters and aligned sparse scene reconstructions from COLMAP.
Key Results
Ablation studies comparing the proposed method against standard baselines showed significant improvements:
- Quantitative Improvements: On standard 3D Gaussian Splatting, the method improved the PSNR from 24.41 to 25.37 and significantly reduced the LPIPS (perceptual blur) from 0.2060 to 0.1505.
- Qualitative Sharpness: Visual results demonstrate that while baseline methods produce "cloudy" or "ghosted" leaf edges, the iterative approach produces sharp boundaries and textures. Most visual gains occur within the first 30 iterations.
Discussion and Future Work
While effective, the iterative nature of the pipeline is computationally expensive, requiring approximately four days on a high-end NVIDIA A6000 GPU for 100 iterations. Additionally, very fine structures like thin stems remain challenging to extract as clean 3D meshes. Future work will focus on refining optical flow estimation and exploring ways to reduce training time to make the pipeline practical for large-scale agricultural use.
Summary updated