SegmentAnyTree: A sensor and platform agnostic deep learning model for tree segmentation using laser scanning data

Maciej Wielgosz, Stefano Puliti, Binbin Xiang, Konrad Schindler, Rasmus Astrup

Abstract

Individual Tree Crown (ITC) segmentation is a critical task in precision forestry for monitoring biomass and biodiversity. However, existing algorithms are often specialized for specific sensors (e.g., airborne vs. terrestrial LiDAR), limiting their transferability. We present SegmentAnyTree, a deep learning framework that achieves platform agnosticism through a novel "synthetic sparsification" augmentation strategy. Our model outperforms current state-of-the-art methods by 20–30% in accuracy and is significantly more computationally efficient, providing a robust solution for multi-sensor forest inventories.

Introduction

In precision forestry, the ability to segment individual trees from LiDAR data is essential for detailed forest management. Traditionally, researchers have relied on platform-specific algorithms. Airborne Laser Scanning (ALS/ULS) methods typically use top-down perspectives and struggle with sparse understory data. In contrast, Terrestrial (TLS) and Mobile (MLS) methods are optimized for high-density, ground-up scans. This fragmentation necessitates constant model re-tuning. SegmentAnyTree addresses this by providing a single, unified model capable of processing data from any LiDAR platform.

Methodology

The SegmentAnyTree architecture is based on PointGroup, a 3D convolutional neural network (CNN) utilizing the Minkowski Engine for sparse data processing. The model employs three parallel prediction heads: semantic segmentation (tree vs. non-tree), offset prediction (pointing to tree centers), and an embedding space to differentiate individual instances.

The core innovation lies in the synthetic sparsification augmentation strategy. By training on high-density scans that are randomly subsampled to various densities (from 1000 down to 10 points/m²), the model learns abstract geometric features of trees that are invariant to point density.

Datasets and Evaluation

The model was validated across several diverse benchmarks:

  • FOR-instance: A global UAV (ULS) dataset.
  • Wytham Woods: A complex TLS dataset of temperate deciduous forests.
  • TreeLearn: An MLS dataset focusing on mature Beech forests.
  • LAUTx: A multi-layered forest inventory used for direct comparison with state-of-the-art (SOTA) methods like Point2Tree and TLS2trees.

Key Results

SegmentAnyTree demonstrated significant improvements over existing methods:

  • Accuracy: On the LAUTx dataset, the model achieved a 20–30% improvement in F1 score and detection rates compared to Point2Tree.
  • Understory Detection: The model showed a marked increase in the detection of trees under 10 meters, which are often missed by traditional canopy-height-model (CHM) based methods.
  • Efficiency: The approach is approximately 10x more computationally efficient than graph-based alternatives.
  • Robustness: Performance remains stable down to a density of 50 points/m², with a sharp decline only occurring at extremely low densities (10 points/m²).

Discussion and Limitations

A key insight from this research is that "sparsity is a feature." Aggressive augmentation with sparse data actually improved the model's performance on high-resolution datasets by forcing it to learn more robust structural representations.

However, challenges remain in complex, multi-layered broadleaved forests where intertwined crowns make segmentation difficult even for human experts. Future work will focus on expanding the training data to include tropical ecosystems and exploring specialized convolutional kernels to better capture vertical tree structures.

Conclusion

SegmentAnyTree represents a significant step toward a universal tool for forest inventory. By breaking the "sensor silo," it enables practitioners to apply a single, high-performance model across various LiDAR platforms, streamlining the path from raw data to actionable forest metrics.

Summary updated

Abstract

This study focuses on advancing individual tree crown (ITC) segmentation in lidar data, developing a sensor- and platform-agnostic deep learning model transferable across a spectrum of dense laser scanning datasets from drone (ULS), to terrestrial (TLS), and mobile (MLS) laser scanning data. In a field where transferability across different data characteristics has been a longstanding challenge, this research marks a step towards versatile, efficient, and comprehensive 3D forest scene analysis. Central to this study is model performance evaluation based on platform type (ULS vs. MLS) and data density. This involved five distinct scenarios, each integrating different combinations of input training data, including ULS, MLS, and their augmented versions through random subsampling, to assess the model's transferability to varying resolutions and efficacy across different canopy layers. The core of the model, inspired by the PointGroup architecture, is a 3D convolutional neural network (CNN) with dedicated prediction heads for semantic and instance segmentation. The model underwent comprehensive validation on publicly available, machine learning-ready point cloud datasets. Additional analyses assessed model adaptability to different resolutions and performance across canopy layers. Our results reveal that point cloud random subsampling is an effective augmentation strategy and improves model performance and transferability. The model trained using the most aggressive augmentation, including point clouds as sparse as 10 points m−2, showed best performance and was found to be transferable to sparse lidar data and boosts detection and segmentation of codominant and dominated trees. Notably, the model showed consistent performance for point clouds with densities >50 points m−2 but exhibited a drop in performance at the sparsest level (10 points m−2), mainly due to increased omission rates. Benchmarking against current state-of-the-art methods revealed boosts of up to 20% in the detection rates, indicating the model's superior performance on multiple open benchmark datasets. Further, our experiments also set new performance baselines for the other public datasets. The comparison highlights the model's superior segmentation skill, mainly due to better detection and segmentation of understory trees below the canopy, with reduced computational demands compared to other recent methods. In conclusion, the present study demonstrates that it is indeed feasible to train a sensor-agnostic model that can handle diverse laser scanning data, going beyond current sensor-specific methodologies. Further, our study sets a new baseline for tree segmentation, especially in complex forest structures. By advancing the state-of-the-art in forest lidar analysis, our work also lays the foundation for future innovations in ecological modeling and forest management.
BibTeX
@inproceedings{wielgosz2024segmentanytree,
  author = {Maciej Wielgosz and Stefano Puliti and Binbin Xiang and Konrad Schindler and Rasmus Astrup},
  title = {{SegmentAnyTree: A sensor and platform agnostic deep learning model for tree segmentation using laser scanning data}},
  booktitle = {Remote},
  year = {2024},
  doi = {10.1016/j.rse.2024.114367},
}
Figure