TL;DR
This paper introduces a novel deep learning approach using Fully Convolutional Networks (FCNs) to efficiently classify Airborne Laser Scanner (ALS) point clouds for both ground extraction (DTM) and multi-class land-cover mapping. By projecting 3D point clouds into 2D images and employing FCNs with dilated kernels, the method achieves high accuracy and significantly reduces computational time compared to existing deep learning techniques.
What Problem Does This Paper Address?
Extracting meaningful information from Airborne Laser Scanner (ALS) point clouds, such as creating Digital Terrain Models (DTMs) and classifying land cover, is crucial for various geospatial applications. However, existing methods often struggle with accuracy and computational efficiency, especially when dealing with large-scale datasets. Traditional ground filtering algorithms can fail in complex terrains or urban environments, and even early deep learning approaches for point cloud classification can be computationally expensive due to inefficient point-to-image conversion. This paper aims to bridge this gap by proposing a more accurate and computationally efficient deep learning framework for both ground and multi-class classification of ALS data.
Methods
The core of the proposed method involves transforming the 3D point cloud data into a 2D image representation, which can then be processed by a Fully Convolutional Network (FCN).
- Point-to-Image Conversion: Instead of converting each point individually into a feature image (which is computationally redundant), the authors project the entire ALS point cloud onto a horizontal plane, creating a multi-dimensional image. For ground classification, this image is based on the lowest point within each pixel, incorporating features like elevation, intensity, return number, and height difference. For multi-class classification, a second set of images is generated using the highest point within each pixel, incorporating features like elevation, intensity, number of returns, and height difference. A Multi-Scale FCN (MS-FCN) architecture is also proposed to mitigate information loss during this conversion by using multiple pixel resolutions (e.g., 1m and 0.5m).
- Fully Convolutional Network (FCN) Classification: The authors adapt an FCN with dilated kernels (FCN-DK) for pixel-wise image classification. This architecture maintains the spatial resolution of feature maps throughout the network, allowing for precise pixel-level predictions. Dilated convolutions are used to increase the receptive field without increasing the number of parameters, enabling the network to capture broader spatial context.
- Label Transfer: After the FCN predicts labels for each pixel, these labels are transferred back to the original ALS points. For ground classification, if a pixel is labeled as ground, the lowest point within that pixel is initially labeled as ground. A threshold-based densification step is then applied to label remaining points. For multi-class classification, a Multi-Task Network (MTN-FCN) is proposed, which uses two FCN blocks to predict both ground/non-ground labels and then further classify non-ground points into finer classes (e.g., vegetation, building).
Datasets
The proposed methods were evaluated on two datasets:
- ISPRS Filter Test Dataset: This is a benchmark dataset for ground classification, consisting of 15 sample areas with varying terrain characteristics. It has a relatively low point density (around 1 point/m²).
- Actueel Hoogtebestand Nederland (AHN3) Dataset: This is a modern, high point-density dataset (8-10 points/m²) covering the Netherlands. It was used for both ground classification and multi-class classification (ground, vegetation, building, water, bridge).
Key Insights and Ideas
- Efficient Point-to-Image Conversion: Projecting the entire point cloud into a single multi-dimensional image, rather than processing each point individually, drastically reduces computational time for data preparation.
- FCNs for Pixel-Wise Classification: Fully Convolutional Networks are well-suited for directly predicting class labels for each pixel in the generated images, effectively transforming point cloud classification into an image classification task.
- Dilated Convolutions for Context: Using dilated convolutions in the FCN architecture allows the network to capture larger spatial contexts without increasing computational complexity or the number of parameters.
- Multi-Scale Approach for Information Preservation: Employing multi-scale images (different pixel sizes) in the MS-FCN architecture helps to better preserve spatial details and improve classification accuracy, especially for small objects.
- Multi-Task Learning for Complex Classification: A Multi-Task Network (MTN-FCN) can effectively handle multiple classification objectives (e.g., ground vs. non-ground, and then non-ground into vegetation/building) within a single framework.
Key Results
- Ground Classification (ISPRS Dataset): The proposed FCN method achieved a lower total error (5.21%) compared to a CNN-based method (7.91%) and LAStools software (12.58%).
- Ground Classification (AHN3 Dataset): The MTN-FCN (used for ground classification) achieved the lowest average total error (2.64%) compared to various FCN configurations and LAStools (3.51%).
- Multi-Class Classification (AHN3 Dataset): The MTN-FCN achieved higher precision (0.93) and recall (0.94) compared to Random Forest (RF) (0.91 precision, 0.92 recall). It also accurately detected small buildings.
- Computational Efficiency: The point-to-image conversion time was reduced from 47 hours to 36 minutes for the ISPRS dataset when using the proposed method compared to a state-of-the-art CNN-based method. Testing time was also significantly reduced.
Limitations and Future Work
- Point-to-Image Conversion Dependency: The method still relies on a point-to-image conversion step, which can be a bottleneck for extremely large datasets.
- Pixel-Level Labeling: For multi-class classification, if multiple non-ground classes are mixed within a single pixel (e.g., a tree branch over a building), only one class label is assigned to all points in that pixel.
- Handling of Large Structures: Very large buildings (over 100m) and flat bridges presented challenges, as their size exceeded the receptive field of the FCN, leading to misclassifications.
- Noise Sensitivity: Outliers in the point cloud can affect the point-to-image conversion and subsequent classification accuracy. Removing outliers improved accuracy by 2.62%.
- Future Work: The authors suggest exploring alternative point-to-image conversion strategies with additional features, or investigating voxelization as an alternative to image-based conversion to better preserve 3D structure and avoid pixel-level limitations. Improving the handling of large structures and noise remains an area for future research.
Summary updated