TL;DR
PANet is a novel framework for LiDAR Panoptic Segmentation that replaces the traditional, computationally expensive "offset branch" with a fast, non-learning Sparse Instance Proposal (SIP) module. By combining this with a KNN-Transformer for instance aggregation, PANet effectively resolves over-segmentation of large objects while achieving state-of-the-art results at significantly higher speeds than previous methods.
Introduction: The Challenge of LiDAR Panoptic Segmentation
LiDAR Panoptic Segmentation (LPS) is a critical task for autonomous driving, requiring simultaneous semantic segmentation (identifying object classes) and instance segmentation (distinguishing individual objects).
Most existing approaches utilize a "clustering-based" method where a neural network predicts an "offset" vector from every point to its object center. However, LiDAR data's inherent sparsity and varying density make accurate offset prediction difficult. A common failure mode is "over-segmentation," where large objects like buses or trucks are mistakenly split into multiple smaller fragments. PANet addresses these issues by rethinking the instance grouping process.
Methodology
PANet introduces a streamlined architecture that avoids the unreliable offset head in favor of two primary components: the Sparse Instance Proposal (SIP) module and the Instance Aggregation (IA) module.
Sparse Instance Proposal (SIP) Module
The SIP module is a non-learning component that groups points into instances through a three-step "sampling-shifting-grouping" workflow:
- Balanced Point-Sampling (BPS): Space is divided into voxels, and one representative "seed point" is selected per voxel. This ensures a uniform point distribution regardless of distance from the sensor, reducing computational overhead.
- Bubble Shrinking (BS): An iterative algorithm that "shrinks" seed points toward cluster centers without requiring a trained network for geometric guidance.
- Connected Component Labeling (CCL): A graph-based algorithm that finalizes the grouping of shifted points into initial instance proposals.
Instance Aggregation (IA) Module
To combat the over-segmentation of large objects, the IA module uses a learning-based KNN-Transformer. This module analyzes the proposals from SIP and calculates "affinities" between them. If the model determines that two fragments (e.g., the front and rear of a long truck) belong to the same entity, it merges them into a single instance.
Experimental Results
The researchers evaluated PANet on the SemanticKITTI and nuScenes benchmarks, yielding several key insights:
- State-of-the-Art Accuracy: PANet achieved a Panoptic Quality (PQ) of 61.7% on the SemanticKITTI validation set, outperforming many complex learning-based clustering methods.
- Superior Efficiency: The non-learning SIP module is approximately 13x faster than the Dynamic Shifting (DS) modules used in prior state-of-the-art models (19.5ms vs. 259.1ms).
- Robustness for Large Objects: The IA module significantly improved performance for large vehicle categories. For instance, the PQ for the "Truck" category increased by 3.1% when aggregation was enabled.
Conclusion
By ditching the traditional offset head for a sparse, proposal-based approach, PANet demonstrates that non-learning geometric grouping can be both more efficient and more robust than fully learned alternatives. While challenges remain in extremely sparse long-range data, PANet sets a new standard for real-time, high-accuracy 3D scene understanding.
Summary updated