Sparse Cross-Scale Attention Network for Efficient LiDAR Panoptic Segmentation

Shuangjie Xu, Rui Wan, Maosheng Ye, Xiaoyi Zou, Tongyi CaoAAAI 2022 · DOI

Abstract

LiDAR panoptic segmentation is critical for autonomous driving, requiring simultaneous semantic labeling and instance differentiation. Current methods often struggle with over-segmenting large objects or merging crowded small instances. We present the Sparse Cross-Scale Attention Network (SCAN), which leverages a global cross-scale attention mechanism and sparse Bird’s-Eye-View (BEV) representations to achieve state-of-the-art accuracy while maintaining real-time inference speeds (12.8 FPS).

Introduction

Autonomous vehicles rely on 3D scene understanding to navigate safely. Panoptic segmentation represents the "holy grail" of this task, requiring models to identify both the class of every point and the specific instance it belongs to. However, two primary hurdles persist: surface aggregation and crowding. LiDAR sensors only capture the outer shells of objects; for large vehicles like trucks, points are often distant from the object's center, leading to over-segmentation. Conversely, small, closely packed objects like pedestrians are frequently merged into single blobs (under-segmentation). SCAN addresses these challenges by embracing sparsity and global context.

Methodology

The SCAN architecture introduces three core innovations designed for efficiency and precision:

  1. Cross-Scale Global Attention (CGA): To capture long-range geometric relationships, SCAN employs a global attention mechanism that aligns features across different voxel scales. This allows the network to "connect" distant points on large objects. To maintain computational feasibility, the model utilizes Generalized Kernelizable Attention (GKA).
  2. Multi-scale Sparse Supervision (MSS): Instead of traditional "hard" labels, SCAN uses "soft" labels representing the class distribution within voxels. This provides a richer gradient for learning fine-grained features.
  3. BEV Sparse Distribution (SD): For centroid prediction, SCAN uses a sparse BEV representation. This avoids the memory waste of dense grids and the "blurring" effect of standard 2D convolutions, aiding in the separation of crowded small objects.
Overview of the SCAN architecture, highlighting the Cross-scale Attention and Sparse Distribution Head.

Experiments and Results

SCAN was evaluated on the SemanticKITTI and nuScenes benchmarks. On SemanticKITTI, SCAN achieved a Panoptic Quality (PQ) of 61.5, securing the top spot on the leaderboard at the time of publication and outperforming previous state-of-the-art models like GP-S3Net.

Crucially, SCAN maintains real-time performance, operating at 12.8 FPS on an NVIDIA 3090. This exceeds the 10Hz threshold typically required for vehicle deployment.

Performance (PQ) vs. Runtime (Hz) comparison on SemanticKITTI. SCAN achieves superior accuracy while maintaining real-time speeds.

Conclusion

SCAN demonstrates that global context and sparsity are not mutually exclusive but can be combined to solve complex 3D segmentation tasks efficiently. While the model is highly sensitive to 3D position encoding, its ability to handle both large-scale structures and crowded small objects makes it a robust candidate for real-world autonomous systems. Future work may explore extending this architecture to 4D temporal tracking or further optimization for edge hardware.

Summary updated

BibTeX
@inproceedings{xu2022sparse,
  author = {Shuangjie Xu and Rui Wan and Maosheng Ye and Xiaoyi Zou and Tongyi Cao},
  title = {{Sparse Cross-Scale Attention Network for Efficient LiDAR Panoptic Segmentation}},
  booktitle = {AAAI Conference on Artificial Intelligence},
  year = {2022},
  doi = {10.1609/aaai.v36i3.20197},
}
Figure