Compression of Conditional Deep Learning Network for Fast and Low Power Mobile Applications

Lijie Li, Yan Zhang, Pengfei Wang2017 · DOI

Abstract

Conditional Deep Learning Network (CDLN) is a structure of convolution neural network with multiple classifiers. CDLN could improve the speed for the task of classification while the module of the network is still too large for mobile devices. To address this issue, a method for compressing CDLN, which is named one-shot whole network compression scheme. In the experiments, the module size and time cost are significantly reduced while the accuracy of the network losses a little.

Introduction

While Deep Learning has revolutionized computer vision, state-of-the-art models like VGGNet or ResNet remain computationally expensive and memory-intensive. This creates a significant barrier for mobile applications where battery life is precious and hardware resources are limited. This paper addresses the challenge of deploying high-performance models on mobile platforms by optimizing Conditional Deep Learning Networks (CDLNs).

A CDLN improves efficiency by adding "early-exit" classifiers to the intermediate layers of a network. If a model is confident about an image early on (e.g., an easy-to-recognize object), it exits the network immediately rather than processing all subsequent layers. However, even with these early exits, the underlying convolutional layers are often too bulky for mobile hardware. The authors propose a "one-shot" compression scheme to shrink these layers without compromising predictive power.

Methodology

The proposed approach combines structural optimization with parameter-level compression through a multi-stage pipeline:

1. CDLN Architecture

The researchers begin with a baseline CNN (such as LeNet-5 or AlexNet) and integrate linear classifiers after the initial convolutional layers. A confidence threshold ($\delta$) is used to determine whether a sample can exit early, effectively categorizing inputs into "easy" (classified early) and "hard" (classified in later stages).

Architecture of a Conditional Deep Learning Network (CDLN) with early-exit classifiers.

2. Tucker-2 Decomposition and Automated Rank Selection

To compress the heavy 4D convolutional kernels, the authors apply Tucker-2 decomposition, which breaks a complex tensor into a smaller core tensor and two factor matrices. A major innovation in this work is the use of Variational Bayesian Matrix Factorization (VBMF) to automatically determine the optimal compression rank. This data-driven approach eliminates the need for manual hyperparameter tuning, providing a "one-shot" solution for network slimming.

Experimental Results

The method was evaluated using the CIFAR-10 dataset on two hardware platforms: a standard laptop (Notebook) and a Meizu MX5 mobile phone. The results demonstrate a massive reduction in resource requirements with minimal impact on accuracy.

Comparison of accuracy, model size, and inference time across different network configurations.

Key Performance Metrics:

  • Model Size Reduction: The AlexNet-based CDLN saw its weights drop from 65M to 12M (an 81.54% reduction), while the LeNet-based model dropped from 37M to 9M (a 75.68% reduction).
  • Inference Speedup: On the MX5 mobile phone, the compressed LeNet-based model was 72.60% faster than its uncompressed counterpart. The compressed AlexNet variant achieved a 58.48% speedup.
  • Accuracy Preservation: The accuracy loss was negligible. For the AlexNet-based model, accuracy only dipped from 66.65% to 66.52%—a loss of just 0.13%.

Conclusion

By combining the structural efficiency of early-exit classifiers with the parameter efficiency of Tucker-2 decomposition and VBMF, this paper presents a viable path for deploying deep learning on low-power devices. The "one-shot" nature of the compression scheme makes it particularly attractive for automated deployment pipelines. Future work may explore the scalability of this approach on higher-resolution datasets like ImageNet and deeper architectures such as DenseNet or MobileNet.

Summary updated

Abstract

is a structure of convolution neural network with multiple classifiers. CDLN could improve the speed for the task of classification while the module of the network is still too large for mobile devices. To address this issue, a method for compressing CDLN, which is named one-shot whole network compression scheme. In the experiments, the module size and time cost are significantly reduced while the accuracy of the network losses a little.
BibTeX
@misc{li2017compression,
  author = {Lijie Li and Yan Zhang and Pengfei Wang},
  title = {{Compression of Conditional Deep Learning Network for Fast and Low Power Mobile Applications}},
  year = {2017},
  doi = {10.2991/icmeit-17.2017.33},
}
Figure