Ground and Multi-Class Classification of Airborne Laser Scanner Point Clouds Using Fully Convolutional Networks

Aldino Rizaldy, Claudio Persello, Caroline Gevaert, Sander Oude Elberink, George VosselmanRemote 2018 · DOI

TL;DR

This paper introduces a novel deep learning approach using Fully Convolutional Networks (FCNs) to efficiently classify Airborne Laser Scanner (ALS) point clouds for both ground extraction (DTM) and multi-class land-cover mapping. By projecting 3D point clouds into 2D images and employing FCNs with dilated kernels, the method achieves high accuracy and significantly reduces computational time compared to existing deep learning techniques.

What Problem Does This Paper Address?

Extracting meaningful information from Airborne Laser Scanner (ALS) point clouds, such as creating Digital Terrain Models (DTMs) and classifying land cover, is crucial for various geospatial applications. However, existing methods often struggle with accuracy and computational efficiency, especially when dealing with large-scale datasets. Traditional ground filtering algorithms can fail in complex terrains or urban environments, and even early deep learning approaches for point cloud classification can be computationally expensive due to inefficient point-to-image conversion. This paper aims to bridge this gap by proposing a more accurate and computationally efficient deep learning framework for both ground and multi-class classification of ALS data.

Methods

The core of the proposed method involves transforming the 3D point cloud data into a 2D image representation, which can then be processed by a Fully Convolutional Network (FCN).

  1. Point-to-Image Conversion: Instead of converting each point individually into a feature image (which is computationally redundant), the authors project the entire ALS point cloud onto a horizontal plane, creating a multi-dimensional image. For ground classification, this image is based on the lowest point within each pixel, incorporating features like elevation, intensity, return number, and height difference. For multi-class classification, a second set of images is generated using the highest point within each pixel, incorporating features like elevation, intensity, number of returns, and height difference. A Multi-Scale FCN (MS-FCN) architecture is also proposed to mitigate information loss during this conversion by using multiple pixel resolutions (e.g., 1m and 0.5m).
  2. Fully Convolutional Network (FCN) Classification: The authors adapt an FCN with dilated kernels (FCN-DK) for pixel-wise image classification. This architecture maintains the spatial resolution of feature maps throughout the network, allowing for precise pixel-level predictions. Dilated convolutions are used to increase the receptive field without increasing the number of parameters, enabling the network to capture broader spatial context.
  3. Label Transfer: After the FCN predicts labels for each pixel, these labels are transferred back to the original ALS points. For ground classification, if a pixel is labeled as ground, the lowest point within that pixel is initially labeled as ground. A threshold-based densification step is then applied to label remaining points. For multi-class classification, a Multi-Task Network (MTN-FCN) is proposed, which uses two FCN blocks to predict both ground/non-ground labels and then further classify non-ground points into finer classes (e.g., vegetation, building).

Datasets

The proposed methods were evaluated on two datasets:

  • ISPRS Filter Test Dataset: This is a benchmark dataset for ground classification, consisting of 15 sample areas with varying terrain characteristics. It has a relatively low point density (around 1 point/m²).
  • Actueel Hoogtebestand Nederland (AHN3) Dataset: This is a modern, high point-density dataset (8-10 points/m²) covering the Netherlands. It was used for both ground classification and multi-class classification (ground, vegetation, building, water, bridge).

Key Insights and Ideas

  • Efficient Point-to-Image Conversion: Projecting the entire point cloud into a single multi-dimensional image, rather than processing each point individually, drastically reduces computational time for data preparation.
  • FCNs for Pixel-Wise Classification: Fully Convolutional Networks are well-suited for directly predicting class labels for each pixel in the generated images, effectively transforming point cloud classification into an image classification task.
  • Dilated Convolutions for Context: Using dilated convolutions in the FCN architecture allows the network to capture larger spatial contexts without increasing computational complexity or the number of parameters.
  • Multi-Scale Approach for Information Preservation: Employing multi-scale images (different pixel sizes) in the MS-FCN architecture helps to better preserve spatial details and improve classification accuracy, especially for small objects.
  • Multi-Task Learning for Complex Classification: A Multi-Task Network (MTN-FCN) can effectively handle multiple classification objectives (e.g., ground vs. non-ground, and then non-ground into vegetation/building) within a single framework.

Key Results

  • Ground Classification (ISPRS Dataset): The proposed FCN method achieved a lower total error (5.21%) compared to a CNN-based method (7.91%) and LAStools software (12.58%).
  • Ground Classification (AHN3 Dataset): The MTN-FCN (used for ground classification) achieved the lowest average total error (2.64%) compared to various FCN configurations and LAStools (3.51%).
  • Multi-Class Classification (AHN3 Dataset): The MTN-FCN achieved higher precision (0.93) and recall (0.94) compared to Random Forest (RF) (0.91 precision, 0.92 recall). It also accurately detected small buildings.
  • Computational Efficiency: The point-to-image conversion time was reduced from 47 hours to 36 minutes for the ISPRS dataset when using the proposed method compared to a state-of-the-art CNN-based method. Testing time was also significantly reduced.

Limitations and Future Work

  • Point-to-Image Conversion Dependency: The method still relies on a point-to-image conversion step, which can be a bottleneck for extremely large datasets.
  • Pixel-Level Labeling: For multi-class classification, if multiple non-ground classes are mixed within a single pixel (e.g., a tree branch over a building), only one class label is assigned to all points in that pixel.
  • Handling of Large Structures: Very large buildings (over 100m) and flat bridges presented challenges, as their size exceeded the receptive field of the FCN, leading to misclassifications.
  • Noise Sensitivity: Outliers in the point cloud can affect the point-to-image conversion and subsequent classification accuracy. Removing outliers improved accuracy by 2.62%.
  • Future Work: The authors suggest exploring alternative point-to-image conversion strategies with additional features, or investigating voxelization as an alternative to image-based conversion to better preserve 3D structure and avoid pixel-level limitations. Improving the handling of large structures and noise remains an area for future research.

Summary updated

Abstract

Various classification methods have been developed to extract meaningful information from Airborne Laser Scanner (ALS) point clouds. However, the accuracy and the computational efficiency of the existing methods need to be improved, especially for the analysis of large datasets (e.g., at regional or national levels). In this paper, we present a novel deep learning approach to ground classification for Digital Terrain Model (DTM) extraction as well as for multi-class land-cover classification, delivering highly accurate classification results in a computationally efficient manner. Considering the top–down acquisition angle of ALS data, the point cloud is initially projected on the horizontal plane and converted into a multi-dimensional image. Then, classification techniques based on Fully Convolutional Networks (FCN) with dilated kernels are designed to perform pixel-wise image classification. Finally, labels are transferred from pixels to the original ALS points. We also designed a Multi-Scale FCN (MS-FCN) architecture to minimize the loss of information during the point-to-image conversion. In the ground classification experiment, we compared our method to a Convolutional Neural Network (CNN)-based method and LAStools software. We obtained a lower total error on both the International Society for Photogrammetry and Remote Sensing (ISPRS) filter test benchmark dataset and AHN-3 dataset in the Netherlands. In the multi-class classification experiment, our method resulted in higher precision and recall values compared to the traditional machine learning technique using Random Forest (RF); it accurately detected small buildings. The FCN achieved precision and recall values of 0.93 and 0.94 when RF obtained 0.91 and 0.92, respectively. Moreover, our strategy significantly improved the computational efficiency of state-of-the-art CNN-based methods, reducing the point-to-image conversion time from 47 h to 36 min in our experiments on the ISPRS filter test dataset. Misclassification errors remained in situations that were not included in the training dataset, such as large buildings and bridges, or contained noisy measurements.
BibTeX
@inproceedings{rizaldy2018ground,
  author = {Aldino Rizaldy and Claudio Persello and Caroline Gevaert and Sander Oude Elberink and George Vosselman},
  title = {{Ground and Multi-Class Classification of Airborne Laser Scanner Point Clouds Using Fully Convolutional Networks}},
  booktitle = {Remote},
  year = {2018},
  doi = {10.3390/rs10111723},
}
Figure