Introduction
Reforestation is a critical component of environmental restoration, particularly in the boreal forests of Alberta, Canada. In these regions, seismic exploration for oil and gas has created thousands of kilometers of narrow clearings that require monitoring to ensure successful regrowth. Traditionally, counting and measuring seedlings is a labor-intensive manual task. While Unmanned Aerial Vehicles (UAVs) and object detection models have streamlined this process, conifer seedlings remain difficult to detect due to their small size and tendency to blend into surrounding vegetation like moss or grass.
This paper demonstrates that integrating height (elevation) data with standard RGB drone imagery significantly improves detection accuracy. Crucially, the researchers found that the method of integration is vital: while adding depth as a fourth image channel (early fusion) is ineffective, injecting it after the feature-extraction backbone (mid-fusion) provides a substantial performance boost.
Methodology
The researchers employed a Faster R-CNN architecture with a ResNet50-FPN backbone. They investigated three primary strategies for fusing height data with visual features:
- Early Fusion ("First"): Height is added as a fourth channel (RGB-D) at the input layer.
- Late Fusion ("Final"): Height data is concatenated to features only at the final classification and bounding-box regression stage.
- Mid-Fusion ("Pre-RPN"): Height information is concatenated to the feature maps after the backbone but before the Region Proposal Network (RPN) and Region of Interest (RoI) heads.
The study utilized a custom dataset of 3,865 image tiles (256x256 pixels) captured at a 30m altitude, yielding a Ground Sampling Distance (GSD) of ~0.75cm. Height data was derived from photogrammetric Digital Surface Models (DSM) and, for specific areas, high-resolution LiDAR-based Canopy Height Models (CHM).
Key Results and Insights
The experimental results highlighted a significant "Backbone Bottleneck." The "First" model (early fusion) often performed worse than the baseline, likely because the ResNet backbone, pre-trained on 3-channel RGB data, could not effectively process the additional depth channel.
In contrast, the Pre-RPN model emerged as the superior architecture. By bypassing the backbone, the geometric height information remains distinct and provides immediate spatial context to the RPN, facilitating better candidate proposals.
As shown in the results table, the Pre-RPN model achieved a Mean Average Precision (MAP) of 0.612 without pre-training, compared to 0.289 for the vanilla RGB model. With transfer learning (pre-training on COCO), the Pre-RPN model's MAP rose to 0.704. Further optimization of the feature map depth for height information (setting it to 32) resulted in a peak MAP of 0.721.
Interestingly, increasing the resolution of height data (comparing SD to HD LiDAR) provided only marginal gains. This suggests that even relatively coarse elevation data is sufficient to help the model distinguish 3D seedlings from the 2D forest floor.
Limitations and Future Work
Despite the improvements, several challenges remain. The current dataset is relatively small, and future work should explore extensive data augmentation to mitigate overfitting. Additionally, while 256x256 tiles were used, larger input sizes might improve Faster R-CNN performance at a higher computational cost.
The authors also suggest investigating "Super-resolution" or depth-estimation networks to predict height from RGB images when true elevation data is unavailable. Finally, addressing class imbalance—specifically the drop in performance when many "negative" (seedling-free) tiles are included—is essential for real-world deployment.
Summary updated