Abstract
Whilst the availability of 3D LiDAR point cloud data has significantly grown in recent years, annotation remains expensive and time-consuming, leading to a demand for semi-supervised semantic segmentation methods with application domains such as autonomous driving. Existing work very often employs relatively large segmentation backbone networks to improve segmentation accuracy, at the expense of computational costs. In addition, many use uniform sampling to reduce ground truth data requirements for learning needed, often resulting in sub-optimal performance. To address these issues, we propose a new pipeline that employs a smaller architecture, requiring fewer ground-truth annotations to achieve superior segmentation accuracy compared to contemporary approaches. This is facilitated via a novel Sparse Depthwise Separable Convolution module that significantly reduces the network parameter count while retaining overall task performance. To effectively sub-sample our training data, we propose a new Spatio-Temporal Redundant Frame Downsampling (ST-RFD) method that leverages knowledge of sensor motion within the environment to extract a more diverse subset of training data frame samples. To leverage the use of limited annotated data samples, we further propose a soft pseudo-label method informed by LiDAR reflectivity. Our method outperforms contemporary semi-supervised work in terms of mIoU, using less labeled data, on the SemanticKITTI (59.5@5%) and ScribbleKITTI (58.1@5%) benchmark datasets, based on a 2.3x reduction in model parameters and 641x fewer multiply-add operations whilst also demonstrating significant performance improvement on limited training data (i.e., Less is More).
Introduction
3D semantic segmentation—the task of assigning a label (like "car," "road," or "tree") to every point in a LiDAR scan—is a cornerstone of autonomous driving. However, the field faces a "complexity trap." Current state-of-the-art models are often massive, requiring millions of parameters and immense computational power. Furthermore, training these models requires vast amounts of human-annotated data, which is incredibly expensive and slow to produce.
Existing semi-supervised methods try to bridge the gap by learning from a mix of labeled and unlabeled data, but they often rely on "dumb" uniform sampling (e.g., labeling every 20th frame) and heavy backbones. This leads to redundant training on similar scenes and high deployment costs. The LiM3D framework addresses these issues by proving that smaller, more efficient models can actually outperform massive architectures in data-scarce environments.
Methodology
The LiM3D pipeline introduces three core innovations to optimize 3D semantic segmentation:
- Sparse Depthwise Separable Convolution (SDSC): Adapting efficient 2D architectures like MobileNet, SDSC splits standard 3D convolutions into spatial "depthwise" steps and "pointwise" channel-mixing steps. This drastically reduces parameter count while retaining the ability to learn complex spatial features.
- Spatio-Temporal Redundant Frame Downsampling (ST-RFD): Instead of uniform sampling, ST-RFD uses sensor motion to identify redundancy. By calculating frame similarity (using SSIM), it selects a diverse subset for labeling, ensuring the model learns from a wide variety of scenarios rather than repeating the same "easy" frames.
- Reflectivity-Informed Soft Pseudo-Labeling: LiM3D leverages LiDAR reflectivity—a physical property of materials—to refine pseudo-labels. Since different materials (e.g., road vs. sidewalk) have distinct reflective signatures, this physical prior helps the model handle unreliable pseudo-labels more effectively through a contrastive loss (InfoNCE).
Experimental Results
The researchers evaluated their approach on two major benchmarks: SemanticKITTI and ScribbleKITTI (a weakly-supervised version with sparse scribbles). They focused on the 5% sampling protocol, simulating real-world constraints where only a tiny fraction of data is manually labeled.
Key results include:
- Accuracy: On SemanticKITTI (at 5% labels), LiM3D achieved 59.5% mIoU, outperforming contemporary semi-supervised methods like LaserMix and 2DPASS.
- Model Complexity: The SDSC module reduced trainable parameters by 57% compared to the standard Cylinder3D backbone.
- Computational Efficiency: The model requires 641x fewer multiply-add operations (MACs) than leading competitors, facilitating real-time deployment on edge hardware.
- Weak Supervision: On ScribbleKITTI, it reached 58.1% mIoU, demonstrating effectiveness even with sparse and noisy labels.
Discussion and Conclusion
The success of LiM3D highlights several key insights. First, efficiency can act as a regularizer; smaller models often generalize better in low-data regimes by avoiding overfitting. Second, physical priors like reflectivity provide significant boosts when geometric data is ambiguous. Finally, temporal diversity in training data is more valuable than raw volume.
While highly effective, future work could address sensor dependency in ST-RFD and further optimize sparse convolution kernels for specific AI accelerators to fully realize the theoretical speedups in real-world hardware.
Summary updated