Learning Point-wise Abstaining Penalty for Point Cloud Anomaly Detection

Shaocong Xu, Pengfei Li, Xinyu Liu, Qianpu Sun, Yang Li, Shihui Guo, Zhen Wang, Bo Jiang, Rui Wang, Kehua Sheng, Bo Zhang, Hao ZhaoarXiv 2023 · arXiv · OpenReview

Abstract

LiDAR-based semantic scene understanding is an important module in the modern autonomous driving perception stack. However, identifying Out-Of-Distribution (OOD) points in a LiDAR point cloud is challenging as point clouds lack semantically rich features when compared with RGB images. We revisit this problem from the perspective of selective classification, which introduces a selective function into the standard closed-set classification setup. Our solution is built upon the basic idea of abstaining from choosing any known categories but learns a point-wise abstaining penalty with a margin-based loss. Synthesizing outliers to approximate unlimited OOD samples is also critical to this idea, so we propose a strong synthesis pipeline that generates outliers originated from various factors: unrealistic object categories, sampling patterns and sizes. We benchmark our method on SemanticKITTI and nuScenes and achieve state-of-the-art results. Risk-coverage analysis further reveals intrinsic properties of different methods.

Introduction

In autonomous driving, LiDAR semantic segmentation models are typically "closed-set," trained to classify points into a fixed number of categories like road, car, or sidewalk. However, real-world environments are "open-set." When a model encounters an object it hasn't seen before—such as furniture fallen onto a roadway—it may dangerously misclassify it as a known category. This failure to identify Out-Of-Distribution (OOD) samples is a significant safety concern.

Comparison of segmentation errors and synthesis methods.

Detecting these outliers is difficult because LiDAR data is geometrically sparse and lacks the rich semantic features (like color or texture) found in images. Previous methods attempted to solve this by synthesizing outliers through simple resizing of existing objects, but this creates "shortcuts" where models identify outliers based on unrealistic point density rather than shape or context.

Methodology

We introduce LiON (Learning Point-wise Abstaining Penalty), which addresses these challenges through two primary innovations:

1. Point-wise Abstaining Penalty

Inspired by selective classification, LiON learns a specific penalty ($\alpha$) for every point. Unlike a fixed global penalty, this point-wise approach allows the model to calibrate its confidence based on local geometry. This is crucial for LiDAR, where point density naturally decreases with distance. The model uses a margin-based loss to encourage low penalties for inliers and high penalties for outliers. LiON is efficient and easy to implement, as it can be added as a simple "head" to existing segmentation architectures like Cylinder3D without requiring complex adversarial training.

2. Diverse ShapeNet Synthesis Pipeline

To train the model to recognize outliers, we developed a realistic synthesis pipeline using the ShapeNet dataset. We integrate diverse 3D models into real LiDAR scenes while accounting for placement, rotation, and sensor-specific sampling patterns. This prevents the density-based "shortcut" problem found in simpler synthesis methods.

Experimental Results

We evaluated LiON on the SemanticKITTI and nuScenes benchmarks, using over 220,000 models from ShapeNet as synthetic outliers.

Performance Benchmarks

LiON significantly outperformed previous state-of-the-art (SOTA) methods across standard metrics like AUPR (Area Under Precision-Recall):

  • SemanticKITTI: Achieved an AUPR of 44.68, compared to the previous best (APF) at 36.10.
  • nuScenes: Achieved an AUPR of 31.58, nearly 50% higher than the previous best (REAL) at 21.20.
Visualization of the segmentation output.

Key Insights

  • The Calibration Trap: We found that some previous methods performed better when their specific calibration losses were removed, suggesting that standard calibration techniques can inadvertently hinder OOD detection.
  • Density Robustness: By learning point-wise penalties, LiON becomes robust to the natural sparsity gradients in LiDAR data, distinguishing between distant inliers and true anomalies.

Conclusion

LiON demonstrates that reframing point cloud anomaly detection as a selective classification problem with point-wise penalties is highly effective. By combining this theoretical approach with a physically-grounded data synthesis pipeline, we establish a new state-of-the-art for safe and reliable LiDAR perception in open-set environments.

Summary updated

Abstract

LiDAR-based semantic scene understanding is an important module in the modern autonomous driving perception stack. However, identifying Out-Of-Distribution (OOD) points in a LiDAR point cloud is challenging as point clouds lack semantically rich features when compared with RGB images. We revisit this problem from the perspective of selective classification, which introduces a selective function into the standard closed-set classification setup. Our solution is built upon the basic idea of abstaining from choosing any known categories but learns a point-wise abstaining penalty with a marginbased loss. Synthesizing outliers to approximate unlimited OOD samples is also critical to this idea, so we propose a strong synthesis pipeline that generates outliers originated from various factors: unrealistic object categories, sampling patterns and sizes. We demonstrate that learning different abstaining penalties, apart from point-wise penalty, for different types of (synthesized) outliers can further improve the performance. We benchmark our method on SemanticKITTI and nuScenes and achieve state-of-the-art results. Risk-coverage analysis further reveals intrinsic properties of different methods. Codes and models will be publicly available.
BibTeX
@misc{xu2023learning,
  author = {Shaocong Xu and Pengfei Li and Xinyu Liu and Qianpu Sun and Yang Li and Shihui Guo and Zhen Wang and Bo Jiang and Rui Wang and Kehua Sheng and Bo Zhang and Hao Zhao},
  title = {{Learning Point-wise Abstaining Penalty for Point Cloud Anomaly Detection}},
  year = {2023},
  eprint = {2309.10230},
  archivePrefix = {arXiv},
  url = {https://arxiv.org/abs/2309.10230},
}
Figure