Abstract
Binarization is an extreme network compression approach that provides large computational speedups along with energy and memory savings, albeit at significant accuracy costs. We investigate the question of where to binarize inputs at layer-level granularity and show that selectively binarizing the inputs to specific layers in the network could lead to significant improvements in accuracy while preserving most of the advantages of binarization. We analyze the binarization tradeoff using a metric that jointly models the input binarization-error and computational cost and introduce an efficient algorithm to select layers whose inputs are to be binarized. Practical guidelines based on insights obtained from applying the algorithm to a variety of models are discussed. Experiments on Imagenet dataset using AlexNet and ResNet-18 models show 3-4% improvements in accuracy over fully binarized networks with minimal impact on compression and computational speed. The improvements are even more substantial on sketch datasets like TU-Berlin, where we match state-of-the-art accuracy as well, getting over 8% increase in accuracies. We further show that our approach can be applied in tandem with other forms of compression that deal with individual layers or overall model compression (e.g., SqueezeNets). Unlike previous quantization approaches, we are able to binarize the weights in the last layers of a network, which often have a large number of parameters, resulting in significant improvement in accuracy over fully binarized models.
Introduction
Deep learning models are notoriously resource-intensive, requiring significant memory and computational power (FLOPs), which limits their deployment on mobile and embedded systems. Binarization—reducing weights and activations to a single bit—offers extreme compression, potentially providing up to 58x speedups and 32x memory savings. However, fully binarized networks often suffer from a significant "performance cliff" where accuracy drops below practical thresholds.
This research addresses the "where to binarize" problem. By recognizing that different layers exhibit varying sensitivities to binarization noise, we propose a data-driven strategy to selectively maintain higher precision in critical layers, thereby recovering accuracy without sacrificing the core benefits of binarization.
Methodology
The creation of Hybrid Binary Networks (HBNs) follows a structured three-step process:
- Joint Metric ($M$): A metric is defined to balance binarization error ($E$) against computational cost ($1/NF$). This allows for the identification of layers where the accuracy loss from binarization outweighs the efficiency gains.
- Partitioning Algorithm: Using a pre-trained fully binarized model, the metric is calculated for each layer. K-means clustering then partitions these layers. Layers in the high-error cluster are designated as WeightBinConv (binary weights, full-precision inputs), while others remain FullBinConv (binary weights and inputs).
- Hybridization Ratio ($R$): This hyperparameter enables tuning the tradeoff between inference speed and model accuracy.
Unlike traditional heuristics that preserve the first and last layers in full precision by default, this automated approach often finds that binarizing the weights of the final layer is effective if its inputs are kept at higher precision, significantly reducing parameter count.
Experimental Results
The framework was evaluated on ImageNet and specialized sketch datasets like TU-Berlin and Sketchy. Sketch datasets are particularly relevant as their inherent sparsity makes them naturally suited for binarized representations.
The hybrid approach demonstrates significant performance improvements across various architectures. On ImageNet, HBNs achieved a 3-4% accuracy boost over fully binarized AlexNet and ResNet-18 models. The gains were even more pronounced on sketch datasets, exceeding 8%. For the Sketch-A-Net model, the hybrid approach achieved a staggering 233x memory compression compared to the full-precision version while matching state-of-the-art accuracy.
Discussion and Insights
- Layer Independence: The authors found that binarization noise does not propagate through the network as heavily as previously feared, allowing for simplified layer-wise optimization.
- Compressibility of the Last Layer: Binarizing weights in the final layer is viable for memory savings when input precision is maintained, challenging the common practice of keeping the last layer in full precision.
- Synergy with Compact Models: When applied to SqueezeNet, the method achieved a 256x overall compression with only a 6% accuracy drop, demonstrating that HBNs can complement existing architectural optimizations.
Conclusion
Hybrid Binary Networks provide a robust solution for optimizing the accuracy-efficiency tradeoff in neural network compression. By selectively binarizing activations based on a joint error-cost metric, HBNs recover significant accuracy lost in fully binarized models while maintaining massive gains in memory and computational efficiency. Future work will explore fine-tuning methods and specialized hardware kernels to further enhance real-world performance on edge devices.
Summary updated