T3DNet: Compressing Point Cloud Models for Lightweight 3D Recognition

Zhiyuan Yang, Yunjiao Zhou, Lihua Xie, Jianfei Yang

Abstract

3D point cloud has been widely used in many mobile application scenarios, including autonomous driving and 3D sensing on mobile devices. However, existing 3D point cloud models tend to be large and cumbersome, making them hard to deploy on edge devices due to their high memory requirements and non-real-time latency. There has been a lack of research on how to compress 3D point cloud models into lightweight models. In this paper, we propose a method called T3DNet (Tiny 3D Network with augmEntation and disTillation) to address this issue. We find that the tiny model after network augmentation is much easier for a teacher to distill. Instead of gradually reducing the parameters through techniques such as pruning or quantization, we pre-define a tiny model and improve its performance through auxiliary supervision from augmented networks and the original model. We evaluate our method on several public datasets, including ModelNet40, ShapeNet, and ScanObjectNN. Our method can achieve high compression rates without significant accuracy sacrifice, achieving state-of-the-art performances on three datasets against existing methods. Amazingly, our T3DNet is 58 times smaller and 54 times faster than the original model yet with only 1.4% accuracy descent on the ModelNet40 dataset.

Introduction: The Challenge of Edge 3D Recognition

3D point clouds are essential for autonomous driving, robotics, and mobile AR, but the deep learning models used to process them are notoriously "heavy." These models require significant memory and computational power, making them nearly impossible to deploy on low-power IoT devices or micro-controllers.

Overview of T3DNet compressing large models for IoT deployment.

While techniques like pruning and quantization work well for 2D images, they often struggle with 3D point clouds. Because point cloud data is inherently sparse and irregular, unstructured compression can break the model's ability to generalize. A specific hurdle identified by the researchers is that "tiny" models (those shrunk to 1/64th the original size) suffer from severe under-fitting. They are simply too small to learn complex geometric features effectively from scratch.

Methods: The T3DNet Framework

T3DNet introduces a structured, two-stage approach to bridge the performance gap between massive "teacher" models and tiny "student" models.

The two-stage T3DNet framework: Tiny network augmentation followed by distillation.

Stage 1: Tiny Network Augmentation (NetAug)

Instead of training the tiny model in isolation, the authors embed it as a "subnet" within a series of larger, augmented networks. During training, they randomly expand the channel sizes of the tiny model's layers. This forces the tiny weights to participate in a higher-capacity representational space, effectively "warming up" the model and mitigating under-fitting.

Stage 2: Tiny Network Distillation (KD)

Once the tiny model has been bolstered by NetAug, it becomes much more receptive to learning. In this stage, the original, full-sized model acts as a teacher, providing "soft labels" (logits) to guide the tiny student. The key innovation is the sequencing: the authors discovered that network augmentation makes the tiny model significantly easier for a teacher to distill compared to training with distillation from the start.

Datasets and Evaluation

The researchers evaluated T3DNet across several standard benchmarks to ensure it works for different 3D tasks:

  • ModelNet40: Synthetic CAD models for 3D object classification.
  • ScanObjectNN: A more challenging, real-world dataset containing noisy, occluded 3D objects.
  • ShapeNet: Used for 3D part segmentation (identifying components like chair legs or airplane wings).
  • ScanNet v2: Large-scale indoor scene scans used to test the framework's generalization on advanced architectures like Point Transformer v2.

Key Insights and Results

The results demonstrate that T3DNet achieves massive compression with minimal performance degradation:

  • Massive Compression: On ModelNet40, they compressed PointNet++ by 58× in parameters (1.74M down to 0.03M) and 54× in FLOPs (31.9G down to 0.6G) with only a 1.45% drop in overall accuracy.
  • The "Teachability" Threshold: A major finding is that tiny models are often too weak to benefit from knowledge distillation immediately. By using NetAug first, the tiny model reaches a baseline level of competence that allows the teacher's "dark knowledge" to actually take hold.
  • Structured Efficiency: By focusing on shrinking channel widths (structured) rather than removing random connections (unstructured), the model maintains a regular architecture that is more efficient for hardware and better for the sparse nature of point clouds.
  • Generalization: The method successfully compressed PointMLP and Point Transformer v2, achieving a 98% parameter reduction on the latter with minimal performance degradation.

Conclusion and Future Work

T3DNet effectively bridges the gap between high-performance 3D models and resource-constrained edge devices. While the two-stage process inherently takes more time to train and requires careful hyperparameter tuning, its results on standard benchmarks are state-of-the-art. Future work will explore applying T3DNet to more complex tasks like 3D object detection in autonomous driving scenes.

Summary updated

Abstract

3D point cloud has been widely used in many mobile application scenarios, including autonomous driving and 3D sensing on mobile devices. However, existing 3D point cloud models tend to be large and cumbersome, making them hard to deploy on edged devices due to their high memory requirements and non-real-time latency. There has been a lack of research on how to compress 3D point cloud models into lightweight models. In this paper, we propose a method called T3DNet (Tiny 3D Network with augmEntation and disTillation) to address this issue. We find that the tiny model after network augmentation is much easier for a teacher to distill. Instead of gradually reducing the parameters through techniques such as pruning or quantization, we pre-define a tiny model and improve its performance through auxiliary supervision from augmented networks and the original model. We evaluate our method on several public datasets, including ModelNet40, ShapeNet, and ScanObjectNN. Our method can achieve high compression rates without significant accuracy sacrifice, achieving state-of-the-art performances on three datasets against existing methods. Amazingly, our T3DNet is 58 times smaller and 54 times faster than the original model yet with only 1.4% accuracy descent on the ModelNet40 dataset.
BibTeX
@article{yang2024t,
  author = {Zhiyuan Yang and Yunjiao Zhou and Lihua Xie and Jianfei Yang},
  title = {{T3DNet: Compressing Point Cloud Models for Lightweight 3D Recognition}},
  journal = {IEEE Transactions on Cybernetics},
  year = {2024},
  doi = {10.1109/tcyb.2024.3487220},
}
Figure