Abstract
The 3-D point cloud has been widely used in many mobile application scenarios, including autonomous driving and 3-D sensing on mobile devices. However, existing 3-D point cloud models tend to be large and cumbersome, making them hard to deploy on edge devices due to their high memory requirements and non-real-time latency. In this article, we propose a method called T3DNet (tiny 3-D network with augmentation and distillation) to address this issue. We find that a tiny model is much easier for a teacher to distill after undergoing network augmentation. Instead of gradually reducing parameters through pruning or quantization, we predefine a tiny model and improve its performance through auxiliary supervision from augmented networks and the original model. Our method achieves high compression rates without significant accuracy sacrifice, reaching state-of-the-art performance on ModelNet40, ShapeNet, and ScanObjectNN. Notably, T3DNet is 58x smaller and 54x faster than the original model with only a 1.4% accuracy descent on ModelNet40.
Introduction
3D point clouds, generated by LiDAR and RGB-D sensors, are essential for autonomous driving and augmented reality. However, the deep learning models used to process them are notoriously heavy, requiring massive memory and high-performance GPUs. While techniques like pruning and quantization work for 2D images, they often struggle with the sparse and irregular nature of 3D data.
Simply training a "tiny" model from scratch usually leads to severe underfitting. T3DNet addresses this by moving away from "shrinking" a large model and instead focuses on "boosting" a pre-defined tiny model through a two-stage process designed for deployment on micro-controllers and mobile devices.
Methodology
T3DNet utilizes a sequential two-stage framework to bridge the performance gap between heavy and lightweight architectures.
Stage 1: Tiny Network Augmentation
To solve the underfitting problem, the tiny model is embedded as a "subnet" within a series of larger, randomly expanded "augmented models." During training, the weights of the tiny model are updated as part of these larger networks. This forces the tiny model to learn more robust and representative geometric features than it would on its own.
Stage 2: Tiny Network Distillation
Once the tiny model has been "warmed up" by augmentation, it undergoes Knowledge Distillation (KD). A large, pre-trained "teacher" model (the original heavy version) guides the tiny "student" model by sharing its "soft labels" (probability distributions). The researchers discovered that this sequential synergy is vital; the augmentation stage makes the student capable enough to effectively learn from the master teacher.
Experiments and Results
The framework was evaluated on several major benchmarks, including ModelNet40 (clean CAD models), ScanObjectNN (noisy real-world data), and ShapeNet (part segmentation).
Key results include:
- Extreme Compression: On ModelNet40, the model size was reduced by 58x (from 1.74M to 0.03M parameters).
- High Speed: Inference is 54x faster than the original baseline.
- Minimal Accuracy Loss: Despite the massive reduction, accuracy only dropped by 1.4% on ModelNet40.
- Robustness: On the noisy ScanObjectNN dataset, T3DNet improved the accuracy of a standalone tiny model by 3.38%, proving its effectiveness with messy sensor data.
Conclusion
T3DNet demonstrates that structured compression, when combined with a sequential augmentation-distillation pipeline, can produce highly efficient 3D models. While the training process is more intensive upfront, the resulting models are exceptionally well-suited for real-time IoT applications. Future work will explore applying T3DNet to complex tasks like 3D object detection in large-scale autonomous driving scenes.
Our code is available at: https://github.com/Zhiyuan002/T3DNet
Summary updated