SEMANTIC LABELING OF ALS POINT CLOUDS FOR TREE SPECIES MAPPING USING THE DEEP NEURAL NETWORK POINTNET++

S. Briechle, Peter Krzystek, George Vosselman2019 · DOI

TL;DR

This paper explores the use of the deep neural network PointNet++ for semantic labeling of airborne laser scanning (ALS) point clouds to map tree species (spruce and beech). While achieving competitive per-point classification results using only geometric features, the deep learning approach slightly underperformed a traditional Random Forest classifier, highlighting the impact of training data quality and edge effects.

What Problem Does This Paper Address?

Accurately mapping tree species is crucial for forest management and ecological studies. Traditional methods often rely on segmenting individual trees and then classifying them using hand-crafted features with classifiers like Random Forests (RF). While these methods can achieve around 90% accuracy for distinguishing coniferous and deciduous trees using just geometric ALS data, there's a drive to leverage the power of deep neural networks (DNNs) for potentially higher accuracy and automated feature learning. However, applying DNNs directly to large-scale, irregular 3D point clouds for this task has been less explored, partly due to the need for large, well-labeled datasets and the challenge of adapting DNN architectures to this specific data type.

Methods

The core of this research is the application of PointNet++, a deep neural network designed to directly process unordered 3D point clouds without requiring prior conversion to voxels or images. PointNet++ employs a hierarchical feature learning approach, allowing it to capture multi-scale information based on local point densities. For this study, the authors adapted PointNet++ for semantic segmentation, specifically tuning hyperparameters for a binary classification task (coniferous vs. deciduous). They also implemented a baseline Random Forest (RF) classifier that uses hand-crafted geometric features for comparison.

Datasets

The study utilizes Airborne Laser Scanning (ALS) data acquired over the Bavarian Forest National Park. This data was characterized by a point density of approximately 54 points/m².

  • Training Data Generation: A preliminary RF classifier was used to automatically label tree segments for training the PointNet++ model. This initial labeling achieved precision/recall scores of 93%/80% for coniferous trees and 82%/92% for deciduous trees.
  • PointNet++ Training and Evaluation: A large area (2.64 km²) with approximately 97,000 tree segments was used for training PointNet++. The evaluation was performed on randomly chosen test batches.

Key Insights and Ideas

  • Direct 3D Point Cloud Processing: PointNet++ demonstrates the feasibility of directly applying advanced deep learning architectures to raw ALS point clouds for semantic labeling, bypassing traditional feature engineering steps.
  • Geometric Features Suffice (to an extent): The study shows that even with only geometric information (height, density, crown shape), PointNet++ can achieve respectable per-point classification accuracy for tree species.
  • Data Quality Matters: The authors acknowledge that errors in the automatically generated training data and "edge effects" (misclassifications at the boundaries of processing blocks) likely limited the performance of PointNet++. This underscores the critical importance of high-quality training data for DNNs.
  • Point-wise vs. Tree-wise Evaluation: A key distinction is made between the per-point classification results of PointNet++ and the per-tree results of the RF baseline. While PointNet++'s per-point metrics are comparable, the overall tree-level performance might differ.

Key Results

  • PointNet++ Performance: After 163 training epochs, PointNet++ achieved per-point classification results of:
  • Coniferous trees: Precision = 90%, Recall = 79%
    • Deciduous trees: Precision = 81%, Recall = 91%
  • Baseline RF Performance: The initial RF classifier used for generating training data achieved:
  • Coniferous trees: Precision = 93%, Recall = 80%
    • Deciduous trees: Precision = 82%, Recall = 92%
  • Comparison: The per-point classification results from PointNet++ were slightly lower than the per-tree results of the baseline RF classifier. The model performed better for coniferous trees than deciduous trees, similar to the RF baseline.

Limitations and Future Work

  • Limitations:
    • Training Data Errors: The presence of misclassified segments in the automatically generated training data impacted performance.
  • Edge Effects: Misclassifications occurred at the boundaries of processing blocks due to the lack of inter-block context.
  • Geometric Features Only: The current model does not incorporate other potentially useful features like laser intensity or surface normals.
  • Future Work:
    • Integrating Additional Features: Incorporating laser intensity, surface normals, and multispectral features into the DNN is expected to improve accuracy.
  • Addressing Edge Effects: Investigating methods like Superpoint Graphs could help mitigate boundary issues.
  • Individual Tree Species Classification: Extending the approach to classify more than two tree species, potentially using pre-segmented individual trees, is a key future direction.

Summary updated

Abstract

Most methods for the mapping of tree species are based on the segmentation of single trees that are subsequently classified using a set of hand-crafted features and an appropriate classifier. The classification accuracy for coniferous and deciduous trees just using airborne laser scanning (ALS) data is only around 90% in case the geometric information of the point cloud is used. As deep neural networks (DNNs) have the ability to adaptively learn features from the underlying data, they have outperformed classic machine learning (ML) approaches on well-known benchmark datasets provided by the robotics, computer vision and remote sensing community. Though, tree species classification using deep learning (DL) procedures has been of minor research interest so far. Some studies have been conducted based on an extensive prior generation of images or voxels from the 3D raw data. Since innovative DNNs directly operate on irregular and unordered 3D point clouds on a large scale, the objective of this study is to exemplarily use PointNet++ for the semantic labeling of ALS point clouds to map deciduous and coniferous trees. The dataset for our experiments consists of ALS data from the Bavarian Forest National Park (366 trees/ha), only including spruces (coniferous) and beeches (deciduous). First, the training data were generated automatically using a classic feature-based Random Forest (RF) approach classifying coniferous trees (precision = 93%, recall = 80%) and deciduous trees (precision = 82%, recall = 92%). Second, PointNet++ was trained and subsequently evaluated using 80 randomly chosen test batches à 400 m2. The achieved per-point classification results after 163 training epochs for coniferous trees (precision = 90%, recall = 79%) and deciduous trees (precision = 81%, recall = 91%) are fairly high considering that only the geometry was included. Nevertheless, the classification results using PointNet++ are slightly lower than those of the baseline method using a RF classifier. Errors in the training data and occurring edge effects limited a better performance. Our first results demonstrate that the architecture of the 3D DNN PointNet++ can successfully be adapted to the semantic labeling of large ALS point clouds to map deciduous and coniferous trees. Future work will focus on the integration of additional features like i.e. the laser intensity, the surface normals and multispectral features into the DNN. Thus, a further improvement of the accuracy of the proposed approach is to be expected. Furthermore, the classification of numerous individual tree species based on pre-segmented single trees should be investigated.
BibTeX
@inproceedings{briechle2019semantic,
  author = {S. Briechle and Peter Krzystek and George Vosselman},
  title = {{SEMANTIC LABELING OF ALS POINT CLOUDS FOR TREE SPECIES MAPPING USING THE DEEP NEURAL NETWORK POINTNET++}},
  booktitle = {˜The},
  year = {2019},
  doi = {10.5194/isprs-archives-xlii-2-w13-951-2019},
}
Figure