TL;DR
This paper introduces a massive, open-source dataset of over 100 million individual tree crowns across 37 sites in the US, derived from remote sensing data using deep learning. This dataset addresses the long-standing challenge of obtaining individual tree information at broad scales, enabling new avenues for ecological research.
What Problem Does This Paper Address?
Understanding forest ecosystems at the individual tree level is crucial for assessing biodiversity, ecosystem services, and forestry applications. However, collecting this data manually across vast areas is prohibitively expensive and logistically challenging. While remote sensing technologies have advanced, translating raw sensor data into actionable information about individual trees has remained a significant technical hurdle. This gap limits our ability to conduct large-scale ecological analyses and compare forest dynamics across different regions.
Methods
The core of this work is a semi-supervised deep learning approach, building upon the "DeepForest" Python package. This workflow utilizes a one-shot object detector with a convolutional neural network backbone to identify tree crowns in RGB imagery. The model was pre-trained on ImageNet and then fine-tuned using a combination of weak labels derived from existing LiDAR tree detection algorithms and hand-annotated crowns from NEON sites. A key refinement in this study is the incorporation of a 3-meter height filter using LiDAR-derived canopy height models to improve accuracy and distinguish trees from smaller vegetation. The output for each detected crown includes its spatial location (bounding box), estimated crown area, an estimate of tree height, and a confidence score.
Datasets
- Training Data:
- ImageNet (for initial pre-training)
- Weak labels generated from a previous LiDAR tree detection algorithm on 30 NEON sites.
- 10,000 hand-annotated tree crowns from seven NEON sites.
- 207 image-annotated bounding boxes (from 21 NEON sites) used for validation.
- Evaluation Data:
- NEON Airborne Observation Platform (AOP) data (RGB and LiDAR) from 37 NEON sites (2018 or 2019).
- NEON Woody Vegetation Structure data (field-collected stem data).
The paper introduces a novel, large-scale dataset of individual tree crowns, which serves as a benchmark for future research in this area.
Key Insights and Ideas
- Bridging Scales: The most significant insight is the successful creation of a dataset that bridges the gap between local, field-based measurements and broad-scale remote sensing, enabling individual-level analyses across continental extents.
- Deep Learning's Power: The study demonstrates the efficacy of deep learning, specifically semi-supervised approaches, in tackling complex remote sensing tasks like individual tree crown detection, even with variations in data quality and forest types.
- LiDAR as a Refinement: Incorporating LiDAR-derived height information proved crucial for improving the accuracy of tree detection, particularly in distinguishing trees from understory vegetation.
- Generalizability: The model shows good generalization across diverse geographic sites and forest conditions, a critical factor for a continental-scale dataset.
Key Results
- The dataset contains 104,675,304 individual tree crowns across 37 NEON sites.
- The workflow achieved a bounding box recall of 79.1% and a precision of 72.6% when compared to image-annotated crowns.
- Adding the LiDAR 3m height filter resulted in a 7.0% increase in precision with a slight decrease in recall (0.2%).
- A linear-mixed model comparing predicted tree heights to field measurements yielded a root mean squared error (RMSE) of 1.73 m.
- Overstory stem recall rate was 69.4% when compared to field-collected stem data.
Limitations and Future Work
- Sun-Exposed Crowns Only: The dataset primarily captures sun-exposed canopy trees, excluding subcanopy trees. This limits its direct applicability to analyses heavily reliant on understory structure.
- Undersegmentation Bias: The model tends to undersegment trees, grouping multiple individuals into a single crown, and may underestimate crown extent. This can impact studies of tree allometry and biomass.
- Image-Based Annotations: The reliance on image-based annotations means that what appears as a single crown in imagery might be multiple trees, or vice-versa, introducing uncertainty.
- Novel Object Detection: The model struggles with objects not present in the training data (e.g., buildings, weather stations), leading to potential false positives.
- Data Fusion Challenges: Combining RGB and LiDAR data directly into the CNN did not yield improvements, possibly due to geographic variation, sparse LiDAR coverage, and lack of joint pre-training.
Future work could involve:
- Developing methods to incorporate subcanopy tree information.
- Improving the model's ability to handle undersegmentation and estimate crown extent more accurately.
- Exploring techniques for detecting and labeling "unknown" objects.
- Investigating advanced data fusion methods and leveraging hyperspectral data for improved species differentiation.
- Iteratively improving the dataset with ongoing assessment and community annotation efforts.
Summary updated