TPC: Transformation-Specific Smoothing for Point Cloud Models

Wenda Chu, Linyi Li, Bo Li

Point clouds act as the "eyes" of autonomous vehicles, generated by LiDAR sensors to help AI systems understand 3D environments. While point cloud models are powerful, they are notoriously fragile. An attacker doesn't need to alter every data point; they can simply apply "semantic transformations"—like a slight rotation or a tapering effect—to trick a self-driving car into misidentifying a vehicle as a table.

Previous attempts to provide certified robustness (a mathematical guarantee that a prediction won't change within a certain range of attack) have struggled. Existing methods rely on linear bound relaxations, which become incredibly "loose" and computationally expensive as the number of points in a cloud increases. This leaves a critical gap where we cannot fully trust these models in safety-critical, real-world scenarios.

TPC (Transformation-Specific Smoothing) is a new framework that provides the first tight and scalable robustness guarantees for point cloud models against semantic attacks. By tailoring randomized smoothing techniques to specific categories of 3D transformations, TPC achieves massive leaps in certified accuracy.

Overview of the TPC framework, which provides certified robustness against semantic transformations like rotation.

How TPC Works

Instead of certifying a standard classifier, TPC creates a "smoothed" version of the model that predicts the most likely class across a distribution of randomly transformed inputs. The core innovation lies in how the authors categorize and handle different 3D transformations mathematically:

  1. Additive Transformations: (e.g., shearing along a fixed axis) Applying two transformations is equivalent to one transformation with the sum of their parameters.
  2. Composable Transformations: (e.g., general 3D rotation) The result of two transformations remains within the same family, even if the underlying math is complex.
  3. Indirectly Composable Transformations: (e.g., tapering) These do not have a simple composition property but can be bounded by comparing them to nearby sampled transformations.
Illustration of the three transformation categories: (a) additive, (b) composable, and (c) indirectly composable.

For complex cases like general rotation, TPC uses a novel sampling-based algorithm. It certifies the model at specific points in the transformation space and uses a new "interpolation error bound" to guarantee robustness for every possible angle in between.

Key Insights

  • Scalability is a Feature, Not a Bug: In most certified defense methods, adding more data points makes certification harder and less accurate. TPC flips this paradigm: as the point cloud size increases, the certified accuracy actually improves because the smoothed model becomes more stable.
  • Geometry-Aware Smoothing: By recognizing that rotations and shears have fundamentally different mathematical properties, the authors derived much tighter bounds than previous "one-size-fits-all" approaches.
  • Beyond $\ell_p$ Noise: Most robustness research focuses on small, imperceptible pixel-level changes ($\ell_p$ perturbations). TPC successfully moves the needle toward semantic robustness, which is far more relevant to how objects physically change or are viewed in the real world.

Performance and Results

The researchers evaluated TPC on standard benchmarks like ModelNet40 (3D object classification) and ShapeNet (part segmentation). TPC significantly outperformed the previous state-of-the-art baseline across almost every metric:

  • Twisting Attacks: For a $z$-axis twist within $\pm20^\circ$, TPC achieved 83.8% certified accuracy, compared to just 20.3% for the baseline.
  • General Rotation: TPC provides strong guarantees for rotations up to $10^\circ$ (69.2% accuracy), a feat prior work couldn't even attempt at scale.
  • Model Size Resilience: On a 1,024-point model, TPC maintained ~90% certified accuracy for small rotations, while the baseline's performance plummeted to ~32% due to relaxation errors.

Limitations and Future Work

The primary drawback of TPC, common to all randomized smoothing methods, is computational overhead. To certify a single point cloud, the model must perform thousands of inferences to estimate the smoothed probability distribution. While batch processing speeds this up (yielding inference times between 3ms and 10ms), it remains slower than a standard single-pass model.

Future work could explore more efficient sampling strategies or investigate how to combine TPC with other defense types to handle hybrid attacks involving both semantic transformations and point-level noise.

Summary updated

Abstract

Point cloud models with neural network architectures have achieved great success and been widely used in safety-critical applications, such as Lidar-based recognition systems in autonomous vehicles...
BibTeX
@inproceedings{chu2022tpc,
  author = {Wenda Chu and Linyi Li and Bo Li},
  title = {{TPC: Transformation-Specific Smoothing for Point Cloud Models}},
  booktitle = {International Conference on Machine Learning},
  year = {2022},
  url = {https://openreview.net/forum?id=18B9uqaRB3d},
}
Figure