Diffusion Models-Based Purification for Common Corruptions on Robust 3D Object Detection

Mumuxin Cai, Xupeng Wang, Ferdous Sohel, Hang Lei

TL;DR

This paper introduces LiDARPure, a novel method that uses diffusion models to clean corrupted LiDAR data for 3D object detection. By treating LiDAR scenes as collections of smaller units and applying a conditional diffusion process, LiDARPure effectively removes various corruptions, significantly improving the robustness of 3D object detectors.

What Problem Does This Paper Address?

LiDAR sensors are crucial for 3D perception in applications like autonomous driving. However, the data they collect is prone to various corruptions (e.g., noise, fog, snowfall, density degradation) that severely degrade the performance of 3D object detection models. Existing defense strategies, like adversarial training, struggle with the scale, sparsity, and variety of these corruptions, often leading to gradient confusion and limited robustness against unknown threats. This paper aims to address this by developing a flexible and effective purification method that can handle a wide range of LiDAR data corruptions without requiring retraining of the detector.

Methods

The core of LiDARPure is the application of diffusion models, specifically a discrete-time conditional diffusion model (similar to DDPM). The process involves:

  • Voxelization/Pillarization: The corrupted LiDAR scene is first divided into smaller, independent units (voxels or pillars). This helps manage the large scale and sparsity of LiDAR data.
  • Forward Diffusion: Noise is progressively added to each unit, transforming the corrupted point cloud into Gaussian noise. This step smooths out corruptions.
  • Reverse Diffusion (Conditional): A neural network (noise predictor) is trained to reverse this process, gradually denoising the data to reconstruct a purified point cloud unit. Crucially, this process is conditional, using latent geometric features extracted from the original (corrupted) unit as guidance. This helps the model learn the underlying scene structure.
  • Denoising Recovery Loss: A novel loss function, Lrec, is introduced, comprising two components:
  • diff: Measures the difference in latent features between the purified and original pure scene.
  • pos: Uses the Hausdorff distance to ensure the geometric information of the point clouds is preserved.
  • Detector Independence: LiDARPure is designed to be a pre-processing step, meaning it can be used with any existing 3D object detector without requiring modifications to the detector itself.

Datasets

  • KITTI dataset: Used for training and evaluation. This is a standard benchmark for autonomous driving research, containing a wealth of LiDAR scene data.

Key Insights and Ideas

  • Diffusion Models for LiDAR Purification: The paper demonstrates that diffusion models, known for their generative capabilities, are highly effective for purifying corrupted LiDAR data, overcoming limitations of previous generative models like GANs and VAEs.
  • Unit-based Processing: Dividing large LiDAR scenes into smaller units (voxels/pillars) is key to managing computational complexity and enabling effective diffusion and denoising.
  • Latent Feature Conditioning: Using latent geometric features as conditions in the reverse diffusion process significantly improves the quality of the purified point clouds, guiding the reconstruction towards the original scene structure.
  • Denoising Recovery Loss: The proposed Lrec loss, combining latent feature and geometric preservation, is crucial for ensuring the purified data is suitable for downstream 3D object detection tasks.
  • Limitations of Adversarial Training: The paper highlights that traditional adversarial training methods are often brittle, struggle with diverse corruptions, and can negatively impact performance on clean data, underscoring the need for more robust purification methods.

Key Results

  • LiDARPure effectively purifies 19 common types of LiDAR data corruption.
  • It significantly improves the average precision (AP) of 3D object detectors by up to 20% when faced with corrupted data, outperforming existing defense strategies.
  • LiDARPure shows minimal impact on the performance of detectors when applied to clean scenes, maintaining high standard AP.
  • Quantitative results across different detectors (PointPillars, PointRCNN, PV-RCNN) and various corruption types consistently show LiDARPure achieving superior robust AP compared to adversarial training methods and other purification techniques. For instance, on average, LiDARPure achieved a robust AP of 68.0% (voxel) and 67.9% (pillar) for PointPillars, significantly higher than other methods.
  • Qualitative results demonstrate LiDARPure's ability to smooth out noise (e.g., from snowfall or Gaussian noise) and recover more points around objects, making them more detectable.

Limitations and Future Work

  • Incomplete Recovery in Empty Regions: LiDARPure, like other methods, cannot recover point clouds in completely empty regions where LiDAR data is inherently missing due to occlusion or sensor limitations.
  • High Computational Cost: The diffusion process, while effective, is computationally intensive, limiting its application in real-time scenarios.
  • Future Work: The authors suggest that the modular design of LiDARPure allows for easy integration into existing pipelines. Future work could focus on optimizing the computational efficiency for real-time applications. They also note that while voxel-based processing excels at noise reduction, pillar-based approaches might be better for handling large-scale corruptions like density decrease and shear.

Summary updated

Abstract

LiDAR sensors have been shown to generate data with various common corruptions, which seriously affect their applications in 3D vision tasks, particularly object detection. At the same time, it has been demonstrated that traditional defense strategies, including adversarial training, are prone to suffering from gradient confusion during training. Moreover, they can only improve their robustness against specific types of data corruption. In this work, we propose LiDARPure, which leverages the powerful generation ability of diffusion models to purify corruption in the LiDAR scene data. By dividing the entire scene into voxels to facilitate the processes of diffusion and reverse diffusion, LiDARPure overcomes challenges induced from adversarial training, such as sparse point clouds in large-scale LiDAR data and gradient confusion. In addition, we utilize the latent geometric features of a scene as a condition to assist the generation of diffusion models. Detailed experiments show that LiDARPure can effectively purify 19 common types of LiDAR data corruption. Further evaluation results demonstrate that it can improve the average precision of 3D object detectors to an extent of 20% in the face of data corruption, much higher than existing defence strategies.
BibTeX
@article{cai2024diffusion,
  author = {Mumuxin Cai and Xupeng Wang and Ferdous Sohel and Hang Lei},
  title = {{Diffusion Models-Based Purification for Common Corruptions on Robust 3D Object Detection}},
  journal = {Sensors},
  year = {2024},
  doi = {10.3390/s24165440},
}
Figure