Lite-HDSeg: LiDAR Semantic Segmentation Using Lite Harmonic Dense Convolutions

Ryan Razani, Ran Cheng, Ehsan Taghavi, Bingbing LiuICRA 2021

Abstract

Autonomous driving vehicles and robotic systems rely on accurate perception of their surroundings. Scene understanding is one of the crucial components of perception modules. Among all available sensors, LiDARs are one of the essential sensing modalities of autonomous driving systems due to their active sensing nature with high resolution of sensor readings. Accurate and fast semantic segmentation methods are needed to fully utilize LiDAR sensors for scene understanding. In this paper, we present Lite-HDSeg, a novel real-time convolutional neural network for semantic segmentation of full 3D LiDAR point clouds. Lite-HDSeg can achieve the best accuracy vs. computational complexity trade-off in SemanticKitti benchmark and is designed on the basis of a new encoder-decoder architecture with light-weight harmonic dense convolutions as its core. Moreover, we introduce ICM, an improved global contextual module to capture multi-scale contextual features, and MCSPN, a multi-class Spatial Propagation Network to further refine the semantic boundaries. Our experimental results show that the proposed method outperforms state-of-the-art semantic segmentation approaches which can run real-time, thus is suitable for robotic and autonomous driving applications.

Introduction

For autonomous vehicles and robots, understanding the surrounding scene is a critical requirement. LiDAR sensors provide high-resolution 3D data, but processing this information in real-time is computationally demanding. Current approaches typically fall into two categories: point-based methods, which are accurate but slow, and projection-based methods, which project 3D points onto a 2D "range-image" for efficient processing. While fast, projection-based methods often struggle with accuracy at object boundaries. Lite-HDSeg bridges this gap, offering the speed of 2D projection with the accuracy typically reserved for heavier models.

Methodology

The authors propose an encoder-decoder architecture optimized for range-images. The architecture integrates several key innovations to maximize efficiency without sacrificing performance.

The Lite-HDSeg encoder-decoder architecture featuring ICM, LHD, and MCSPN modules.

Lite Harmonic Dense Convolutions (LHD)

Inspired by HarDNet, the authors developed a "Lite" version of the Harmonic Dense Block. By systematically pruning redundant skip connections, they reduced computational complexity and memory traffic. This design prevents the accumulation of redundant features, leading to a model that is both faster and more accurate than standard dense architectures.

Inception-like Context Module (ICM)

Placed at the beginning of the network, the ICM uses multi-scale kernels and dilated convolutions to capture both local details and global 360-degree context. This allows the encoder to process features with a better understanding of the overall scene geometry from the start.

Multi-class Spatial Propagation Network (MCSPN)

To address the blurring of boundaries common in projection methods, the MCSPN module refines semantic masks before the final output. It learns affinities between neighboring pixels to ensure that object boundaries remain sharp and consistent.

Hybrid Loss Function

The model is trained using a combination of Weighted Cross-Entropy, Lovász-Softmax (to optimize mIoU directly), and a dedicated Boundary Loss to focus the network on difficult edges between classes.

Experimental Results

The model was evaluated on SemanticKITTI, a large-scale benchmark for LiDAR semantic segmentation containing over 43,000 labeled scans.

Accuracy and Efficiency

Lite-HDSeg achieved a 63.8% mIoU on the SemanticKITTI test set, significantly outperforming other real-time projection-based methods like SalsaNext (59.5%) and RangeNet53++ (52.2%).

Performance comparison on SemanticKITTI: Lite-HDSeg achieves the best mIoU vs. Runtime trade-off.

As shown in the performance analysis, Lite-HDSeg occupies the optimal "good performance" region, running at 20 Hz (50ms per scan) on a single NVIDIA GTX 2080Ti GPU. This speed is ideal for autonomous driving applications where sensors typically operate at 10-20 Hz. Ablation studies highlighted that the MCSPN module provided the single largest boost to performance, increasing mIoU by 2.3%.

Conclusion

Lite-HDSeg demonstrates that efficient 2D projections can achieve high-fidelity 3D semantic segmentation. By optimizing dense convolutions and incorporating early global context and boundary refinement, the model sets a new standard for real-time performance. Future work may extend these architectural principles to instance segmentation and 3D object detection tasks.

Summary updated

Abstract

Autonomous driving vehicles and robotic systems rely on accurate perception of their surroundings. Scene understanding is one of the crucial components of perception modules. Among all available sensors, LiDARs are one of the essential sensing modalities of autonomous driving systems due to their active sensing nature with high resolution of sensor readings. Accurate and fast semantic segmentation methods are needed to fully utilize LiDAR sensors for scene understanding. In this paper, we present Lite-HDSeg, a novel real-time convolutional neural network for semantic segmentation of full $3$D LiDAR point clouds. Lite-HDSeg can achieve the best accuracy vs. computational complexity trade-off in SemanticKitti benchmark and is designed on the basis of a new encoder-decoder architecture with light-weight harmonic dense convolutions as its core. Moreover, we introduce ICM, an improved global contextual module to capture multi-scale contextual features, and MCSPN, a multi-class Spatial Propagation Network to further refine the semantic boundaries. Our experimental results show that the proposed method outperforms state-of-the-art semantic segmentation approaches which can run real-time, thus is suitable for robotic and autonomous driving applications.
BibTeX
@inproceedings{razani2021lite,
  author = {Ryan Razani and Ran Cheng and Ehsan Taghavi and Bingbing Liu},
  title = {{Lite-HDSeg: LiDAR Semantic Segmentation Using Lite Harmonic Dense Convolutions}},
  booktitle = {IEEE International Conference on Robotics and Automation},
  year = {2021},
}
Figure