Abstract
LiDAR panoptic segmentation is critical for autonomous driving, requiring simultaneous semantic labeling and instance differentiation. Current methods often struggle with over-segmenting large objects or merging crowded small instances. We present the Sparse Cross-Scale Attention Network (SCAN), which leverages a global cross-scale attention mechanism and sparse Bird’s-Eye-View (BEV) representations to achieve state-of-the-art accuracy while maintaining real-time inference speeds (12.8 FPS).
Introduction
Autonomous vehicles rely on 3D scene understanding to navigate safely. Panoptic segmentation represents the "holy grail" of this task, requiring models to identify both the class of every point and the specific instance it belongs to. However, two primary hurdles persist: surface aggregation and crowding. LiDAR sensors only capture the outer shells of objects; for large vehicles like trucks, points are often distant from the object's center, leading to over-segmentation. Conversely, small, closely packed objects like pedestrians are frequently merged into single blobs (under-segmentation). SCAN addresses these challenges by embracing sparsity and global context.
Methodology
The SCAN architecture introduces three core innovations designed for efficiency and precision:
- Cross-Scale Global Attention (CGA): To capture long-range geometric relationships, SCAN employs a global attention mechanism that aligns features across different voxel scales. This allows the network to "connect" distant points on large objects. To maintain computational feasibility, the model utilizes Generalized Kernelizable Attention (GKA).
- Multi-scale Sparse Supervision (MSS): Instead of traditional "hard" labels, SCAN uses "soft" labels representing the class distribution within voxels. This provides a richer gradient for learning fine-grained features.
- BEV Sparse Distribution (SD): For centroid prediction, SCAN uses a sparse BEV representation. This avoids the memory waste of dense grids and the "blurring" effect of standard 2D convolutions, aiding in the separation of crowded small objects.
Experiments and Results
SCAN was evaluated on the SemanticKITTI and nuScenes benchmarks. On SemanticKITTI, SCAN achieved a Panoptic Quality (PQ) of 61.5, securing the top spot on the leaderboard at the time of publication and outperforming previous state-of-the-art models like GP-S3Net.
Crucially, SCAN maintains real-time performance, operating at 12.8 FPS on an NVIDIA 3090. This exceeds the 10Hz threshold typically required for vehicle deployment.
Conclusion
SCAN demonstrates that global context and sparsity are not mutually exclusive but can be combined to solve complex 3D segmentation tasks efficiently. While the model is highly sensitive to 3D position encoding, its ability to handle both large-scale structures and crowded small objects makes it a robust candidate for real-world autonomous systems. Future work may explore extending this architecture to 4D temporal tracking or further optimization for edge hardware.
Summary updated