Iterative Motion Compensation for Canonical 3D Reconstruction from UAV Plant Images Captured in Windy Conditions

Andre Rochow, Jonas Marcic, Svetlana Seliunina, Sven BehnkearXiv 2025 · DOI · arXiv

Introduction

3D plant phenotyping—measuring physical traits like leaf area or stem height—is vital for modern agriculture. While Unmanned Ground Vehicles (UGVs) can capture stable images, they are expensive and limited by terrain. Drones (UAVs) are a cost-effective alternative, but they face a major technical hurdle: motion.

The Challenge of Motion in the Field

Standard 3D reconstruction algorithms, such as Neural Radiance Fields (NeRF) or 3D Gaussian Splatting (GS), assume the scene is perfectly still. However, in a field environment, wind and the drone's own "downwash" (the air pushed down by the rotors) cause leaves to flutter and bend. This violation of the "static scene" assumption results in blurry, noisy 3D models where fine details are lost. While dynamic reconstruction methods exist, they typically model motion over time. This research instead seeks a "canonical" reconstruction—a single, sharp, motion-compensated snapshot of the plant.

Methodology

The proposed solution consists of an autonomous capture system and an iterative reconstruction pipeline.

Autonomous Capture

Using a DJI Mini 3 Pro and a custom Android application, the drone flies autonomous circles around a plant. To solve the difficult problem of precise localization in a field, the system uses ArUco fiducial markers placed around the plant and an Extended Kalman Filter (EKF) to estimate the drone's pose.

System architecture for autonomous UAV data collection and pose estimation.

Iterative Motion Compensation

The core innovation is an iterative pipeline that progressively warps input images toward a motion-free state. By using the initial (blurry) reconstruction as a spatial anchor and warping images toward this "average" position, the algorithm iteratively converges on a consistent, static geometry.

Visualization of the circular UAV flight path and camera poses relative to the target plant.

The process follows five key steps:

  1. Initial Training: Train a 3D model (e.g., 3D Gaussian Splatting) using the raw, moving images.
  2. Prediction: Render "predicted" images from the 3D model using the original camera viewpoints.
  3. Flow Estimation: Use RAFT (a deep learning model for optical flow) to calculate the displacement between the sharp (but moved) original images and the blurry (but "average" position) rendered images.
  4. Warping: Warp the original images using this flow to align them with the "canonical" 3D representation.
  5. Iteration: Repeat the process. Each iteration reduces scene motion, leading to a sharper 3D model.

Dataset and Results

The researchers are releasing a new dataset featuring 19 individual plants across various crops (Bean, Corn, Soy, and Sugar Beet) and 86 total scenes captured at different growth stages. The data includes camera parameters and aligned sparse scene reconstructions from COLMAP.

Key Results

Ablation studies comparing the proposed method against standard baselines showed significant improvements:

  • Quantitative Improvements: On standard 3D Gaussian Splatting, the method improved the PSNR from 24.41 to 25.37 and significantly reduced the LPIPS (perceptual blur) from 0.2060 to 0.1505.
  • Qualitative Sharpness: Visual results demonstrate that while baseline methods produce "cloudy" or "ghosted" leaf edges, the iterative approach produces sharp boundaries and textures. Most visual gains occur within the first 30 iterations.

Discussion and Future Work

While effective, the iterative nature of the pipeline is computationally expensive, requiring approximately four days on a high-end NVIDIA A6000 GPU for 100 iterations. Additionally, very fine structures like thin stems remain challenging to extract as clean 3D meshes. Future work will focus on refining optical flow estimation and exploring ways to reduce training time to make the pipeline practical for large-scale agricultural use.

Summary updated

Abstract

3D phenotyping of plants plays a crucial role for understanding plant growth, yield prediction, and disease control. We present a pipeline capable of generating high-quality 3D reconstructions of individual agricultural plants. To acquire data, a small commercially available UAV captures images of a selected plant. Apart from placing ArUco markers, the entire image acquisition process is fully autonomous, controlled by a self-developed Android application running on the drone's controller. The reconstruction task is particularly challenging due to environmental wind and downwash of the UAV. Our proposed pipeline supports the integration of arbitrary state-of-the-art 3D reconstruction methods. To mitigate errors caused by leaf motion during image capture, we use an iterative method that gradually adjusts the input images through deformation. Motion is estimated using optical flow between the original input images and intermediate 3D reconstructions rendered from the corresponding viewpoints. This alignment gradually reduces scene motion, resulting in a canonical representation. After a few iterations, our pipeline improves the reconstruction of state-of-the-art methods and enables the extraction of high-resolution 3D meshes. We will publicly release the source code of our reconstruction pipeline. Additionally, we provide a dataset consisting of multiple plants from various crops, captured across different points in time.
BibTeX
@misc{rochow2025iterative,
  author = {Andre Rochow and Jonas Marcic and Svetlana Seliunina and Sven Behnke},
  title = {{Iterative Motion Compensation for Canonical 3D Reconstruction from UAV Plant Images Captured in Windy Conditions}},
  year = {2025},
  eprint = {2510.15491},
  archivePrefix = {arXiv},
  url = {https://arxiv.org/abs/2510.15491},
}
Figure