Less is More: Reducing Task and Model Complexity for 3D Point Cloud Semantic Segmentation

Li Li, Hubert P. H. Shum, Toby P. Breckon

Abstract

The "Less is More" (LiM3D) framework addresses the high computational and annotation costs of 3D LiDAR segmentation. By introducing sparse depthwise separable convolution (SDSC), motion-aware frame sampling (ST-RFD), and reflectivity-informed pseudo-labeling, the authors achieve state-of-the-art accuracy with significantly reduced model complexity, using 2.3× fewer parameters and 641× fewer computations than contemporary models.

Introduction

3D semantic segmentation serves as the "eyes" of autonomous vehicles, enabling them to distinguish between roads, pedestrians, and vehicles using LiDAR point clouds. However, the field currently faces a "complexity crisis" characterized by high annotation costs, model bloat, and data redundancy. Labeling 3D point clouds is notoriously slow and expensive, while state-of-the-art models often rely on massive architectures that are difficult to deploy on edge hardware. Furthermore, uniform frame sampling often leads to redundant training on static scenes, causing overfitting and wasted compute. The LiM3D framework proposes a semi-supervised approach to do more with less labeled data and reduced model complexity.

Methodology

The LiM3D framework introduces three primary innovations to address the challenges of efficiency and data scarcity:

The LiM3D framework pipeline, including training, pseudo-labeling, distillation, and unreliable learning modules.
  1. Sparse Depthwise Separable Convolution (SDSC): Inspired by MobileNet, the authors adapt depthwise separable convolutions for 3D sparse data. By splitting standard 3D convolutions into spatial "depthwise" and "pointwise" steps, they reduce parameter counts by approximately 50% without sacrificing accuracy.
  2. Spatio-Temporal Redundant Frame Downsampling (ST-RFD): This strategy uses vehicle motion to select the most informative frames. By calculating similarity between adjacent images (using SSIM) as a proxy for LiDAR redundancy, it samples more frames during rapid scene changes and fewer when the vehicle is stationary.
  3. Reflectivity-Informed Soft Pseudo-Labeling: LiM3D improves semi-supervised learning by incorporating LiDAR reflectivity, which represents material properties. Furthermore, instead of discarding "unreliable" (high-entropy) predictions, the framework stores them in a category-wise memory bank to serve as negative samples in a contrastive loss function.

Experiments and Results

The method was evaluated on SemanticKITTI (a large-scale dataset with 20 semantic classes) and ScribbleKITTI (a weakly annotated version where only 8% of points are labeled).

Performance comparison of LiM3D against state-of-the-art models in terms of mIoU vs. parameters and Multiply-Adds.

Key results include:

  • Superior Accuracy: On SemanticKITTI (using only 5% of labels), LiM3D achieved 59.5% mIoU, significantly outperforming larger models like Cylinder3D and LaserMix.
  • Massive Efficiency Gains: The SDSC module enabled a 2.3× reduction in model parameters and a staggering 641× reduction in Multiply-Add operations (MACs) compared to baseline models.
  • Sampling Intelligence: The ST-RFD strategy provided a consistent boost of ~1.0% mIoU over uniform sampling by ensuring the training set was maximally diverse.

Discussion and Conclusion

LiM3D demonstrates that efficiency can act as a regularizer; smaller architectures like SDSC are actually less prone to overfitting when training on small data subsets compared to their heavyweight counterparts. A key insight is the "utility of uncertainty," where unreliable predictions are leveraged as negative samples to better define class boundaries. Additionally, using reflectivity as a material proxy helps the model generalize even when labels are scarce.

Future work aims to reduce the dependency on RGB camera images for the ST-RFD sampling strategy and extend SDSC efficiency gains to point-based or graph-based networks. Further optimization for real-time inference latency on low-power automotive chips remains a priority for deployment.

Summary updated

Abstract

Whilst the availability of 3D LiDAR point cloud data has significantly grown in recent years, annotation remains expensive and time-consuming, leading to a demand for semi-supervised semantic segmentation methods with application domains such as autonomous driving. Existing work very often employs relatively large segmentation backbone networks to improve segmentation accuracy, at the expense of computational costs. In addition, many use uniform sampling to reduce ground truth data requirements for learning needed, often resulting in sub-optimal performance. To address these issues, we propose a new pipeline that employs a smaller architecture, requiring fewer ground-truth annotations to achieve superior segmentation accuracy compared to contemporary approaches. This is facilitated via a novel Sparse Depthwise Separable Convolution module that significantly reduces the network parameter count while retaining overall task performance. To effectively sub-sample our training data, we propose a new Spatio-Temporal Redundant Frame Downsampling (ST-RFD) method that leverages knowledge of sensor motion within the environment to extract a more diverse subset of training data frame samples. To leverage the use of limited annotated data samples, we further propose a soft pseudo-label method informed by LiDAR reflectivity. Our method outperforms contemporary semi-supervised work in terms of mIoU, using less labeled data, on the SemanticKITTI (59.5@5%) and ScribbleKITTI (58.1@5%) benchmark datasets, based on a 2.3x reduction in model parameters and 641x fewer multiply-add operations whilst also demonstrating significant performance improvement on limited training data (i.e., Less is More).
BibTeX
@inproceedings{li2023less,
  author = {Li Li and Hubert P. H. Shum and Toby P. Breckon},
  title = {{Less is More: Reducing Task and Model Complexity for 3D Point Cloud Semantic Segmentation}},
  booktitle = {IEEE/CVF Conference on Computer Vision and Pattern Recognition},
  year = {2023},
  doi = {10.1109/cvpr52729.2023.00903},
}
Figure