T3DNet: Compressing Point Cloud Models for Lightweight 3-D Recognition

Zhiyuan Yang, Yunjiao Zhou, Lihua Xie, Jianfei Yang

Abstract

The 3-D point cloud has been widely used in many mobile application scenarios, including autonomous driving and 3-D sensing on mobile devices. However, existing 3-D point cloud models tend to be large and cumbersome, making them hard to deploy on edge devices due to their high memory requirements and non-real-time latency. In this article, we propose a method called T3DNet (tiny 3-D network with augmentation and distillation) to address this issue. We find that a tiny model is much easier for a teacher to distill after undergoing network augmentation. Instead of gradually reducing parameters through pruning or quantization, we predefine a tiny model and improve its performance through auxiliary supervision from augmented networks and the original model. Our method achieves high compression rates without significant accuracy sacrifice, reaching state-of-the-art performance on ModelNet40, ShapeNet, and ScanObjectNN. Notably, T3DNet is 58x smaller and 54x faster than the original model with only a 1.4% accuracy descent on ModelNet40.

Introduction

3D point clouds, generated by LiDAR and RGB-D sensors, are essential for autonomous driving and augmented reality. However, the deep learning models used to process them are notoriously heavy, requiring massive memory and high-performance GPUs. While techniques like pruning and quantization work for 2D images, they often struggle with the sparse and irregular nature of 3D data.

Overview of T3DNet compressing large point cloud models for IoT deployment.

Simply training a "tiny" model from scratch usually leads to severe underfitting. T3DNet addresses this by moving away from "shrinking" a large model and instead focuses on "boosting" a pre-defined tiny model through a two-stage process designed for deployment on micro-controllers and mobile devices.

Methodology

T3DNet utilizes a sequential two-stage framework to bridge the performance gap between heavy and lightweight architectures.

The two-stage training pipeline of T3DNet: network augmentation followed by knowledge distillation.

Stage 1: Tiny Network Augmentation

To solve the underfitting problem, the tiny model is embedded as a "subnet" within a series of larger, randomly expanded "augmented models." During training, the weights of the tiny model are updated as part of these larger networks. This forces the tiny model to learn more robust and representative geometric features than it would on its own.

Stage 2: Tiny Network Distillation

Once the tiny model has been "warmed up" by augmentation, it undergoes Knowledge Distillation (KD). A large, pre-trained "teacher" model (the original heavy version) guides the tiny "student" model by sharing its "soft labels" (probability distributions). The researchers discovered that this sequential synergy is vital; the augmentation stage makes the student capable enough to effectively learn from the master teacher.

Experiments and Results

The framework was evaluated on several major benchmarks, including ModelNet40 (clean CAD models), ScanObjectNN (noisy real-world data), and ShapeNet (part segmentation).

Key results include:

  • Extreme Compression: On ModelNet40, the model size was reduced by 58x (from 1.74M to 0.03M parameters).
  • High Speed: Inference is 54x faster than the original baseline.
  • Minimal Accuracy Loss: Despite the massive reduction, accuracy only dropped by 1.4% on ModelNet40.
  • Robustness: On the noisy ScanObjectNN dataset, T3DNet improved the accuracy of a standalone tiny model by 3.38%, proving its effectiveness with messy sensor data.

Conclusion

T3DNet demonstrates that structured compression, when combined with a sequential augmentation-distillation pipeline, can produce highly efficient 3D models. While the training process is more intensive upfront, the resulting models are exceptionally well-suited for real-time IoT applications. Future work will explore applying T3DNet to complex tasks like 3D object detection in large-scale autonomous driving scenes.

Our code is available at: https://github.com/Zhiyuan002/T3DNet

Summary updated

Abstract

The 3-D point cloud has been widely used in many mobile application scenarios, including autonomous driving and 3-D sensing on mobile devices. However, existing 3-D point cloud models tend to be large and cumbersome, making them hard to deploy on edged devices due to their high memory requirements and nonreal-time latency. There has been a lack of research on how to compress 3-D point cloud models into lightweight models. In this article, we propose a method called T3DNet (tiny 3-D network with augmentation and distillation) to address this issue. We find that the tiny model after network augmentation is much easier for a teacher to distill. Instead of gradually reducing the parameters through techniques, such as pruning or quantization, we predefine a tiny model and improve its performance through auxiliary supervision from augmented networks and the original model. We evaluate our method on several public datasets, including ModelNet40, ShapeNet, and ScanObjectNN. Our method can achieve high compression rates without significant accuracy sacrifice, achieving state-of-the-art performances on three datasets against existing methods. Amazingly, our T3DNet is 58 smaller and 54 faster than the original model yet with only 1.4 accuracy descent on the ModelNet40 dataset. Our code is available at https://github.com/Zhiyuan002/T3DNet.
BibTeX
@article{yang2024t,
  author = {Zhiyuan Yang and Yunjiao Zhou and Lihua Xie and Jianfei Yang},
  title = {{T3DNet: Compressing Point Cloud Models for Lightweight 3-D Recognition}},
  journal = {IEEE Transactions on Cybernetics},
  year = {2024},
  doi = {10.1109/tcyb.2024.3487220},
}
Figure