BMXNet: An Open-Source Binary Neural Network Implementation Based on MXNet

Haojin Yang, Martin Fritzsche, Christian Bartz, Christoph Meinel

TL;DR

This paper introduces BMXNet, an open-source library for Binary Neural Networks (BNNs) built on MXNet. BMXNet enables significant reductions in model size and computational cost by using bit-wise operations, making it easier to deploy powerful deep learning models on resource-constrained devices.

What Problem Does This Paper Address?

Modern deep learning models achieve impressive results but are notoriously computationally expensive and require substantial memory. This makes them challenging to deploy on devices with limited power and processing capabilities, such as mobile phones, wearables, and IoT devices. Binary Neural Networks (BNNs) offer a promising solution by drastically reducing memory footprint and computational complexity through the use of bit-wise operations instead of standard arithmetic. However, implementing and efficiently training these BNNs can be complex.

Methods

BMXNet addresses this by providing an open-source library built on the MXNet deep learning framework. Key features include:

  • Quantized and Binarized Layers: BMXNet introduces QActivation, QConvolution, and QFullyConnected layers that support quantization from 2 to 31 bits, with a special focus on 1-bit binarization.
  • XNOR and Popcount Operations: For binarized layers, BMXNet leverages CPU instructions like XNOR and population count (popcount) to perform matrix multiplications efficiently. This replaces costly floating-point operations with much faster bitwise operations.
  • MXNet Integration: The BNN layers are designed as drop-in replacements for standard MXNet layers, allowing for seamless integration into existing MXNet workflows.
  • GPU and CPU Support: The library supports both GPU and CPU execution, with optimized kernels for efficient computation.
  • Model Converter: A utility is provided to convert trained BNN models into a compact 1-bit format, further reducing storage requirements.

Datasets

The paper evaluates BMXNet on the following datasets:

  • MNIST: A dataset of handwritten digits.
  • CIFAR-10: A dataset of small color images in 10 classes.
  • ImageNet: A large-scale dataset of images for object recognition.

Key Insights and Ideas

  • Efficiency Gains are Substantial: By replacing floating-point operations with bitwise XNOR and popcount, BMXNet achieves significant speedups (up to 125x compared to naive GEMM and 50x compared to Cblas) in convolutional and fully connected layers.
  • Memory Reduction is Dramatic: Binarizing weights allows storing 32 or 64 weights in a single 32-bit or 64-bit value, leading to massive memory savings. A ResNet-18 model, for instance, was compressed by 29x.
  • Accuracy Can Be Maintained: Despite the aggressive quantization to 1 bit, BMXNet demonstrates that competitive accuracy can be achieved, especially when avoiding binarization for the first and last layers of a network.
  • Seamless Integration is Key: Designing BNN layers as drop-in replacements for MXNet layers greatly lowers the barrier to entry for researchers and developers wanting to experiment with BNNs.

Key Results

  • Speedup: BMXNet's xnor_64_omp kernel achieved approximately 50x acceleration compared to Cblas (Atlas) and 125x compared to a naive GEMM kernel for convolutional layers. Even with input binarization, a 13x acceleration was observed compared to Cblas.
  • Model Size Reduction: A ResNet-18 model was compressed by 29x, reducing its size from 44.7MB to 1.5MB.
  • Classification Accuracy:
    • On MNIST, a binary LeNet achieved 0.97 accuracy (vs. 0.99 for full precision).
  • On CIFAR-10, a binary ResNet-18 achieved 0.86 accuracy (vs. 0.90 for full precision).
  • For ImageNet, a partially binarized ResNet-18 with specific full precision stages showed significant accuracy improvements with only minor model size increases.

Limitations and Future Work

  • Accuracy Trade-off: While competitive, there is still a slight accuracy drop compared to full-precision models, particularly for more complex tasks like ImageNet. The paper notes that avoiding binarization for the first and last layers helps mitigate this.
  • Training Complexity: While BMXNet simplifies BNN implementation, training BNNs can still be more sensitive to hyperparameters and optimization strategies than training standard neural networks.
  • Future Work: The authors suggest further optimizations for the xnor GEMM kernel, exploring arbitrary bit widths for weights and activations, and developing more advanced training strategies for BNNs. They also highlight the potential for BMXNet in real-time applications on embedded devices.

Summary updated

Abstract

Binary Neural Networks (BNNs) can drastically reduce memory size and accesses by applying bit-wise operations instead of standard arithmetic operations. Therefore it could significantly improve the efficiency and lower the energy consumption at runtime, which enables the application of state-of-the-art deep learning models on low power devices. BMXNet is an open-source BNN library based on MXNet, which supports both XNOR-Networks and Quantized Neural Networks. The developed BNN layers can be seamlessly applied with other standard library components and work in both GPU and CPU mode. BMXNet is maintained and developed by the multimedia research group at Hasso Plattner Institute and released under Apache license. Extensive experiments validate the efficiency and effectiveness of our implementation. The BMXNet library, several sample projects, and a collection of pre-trained binary deep models are available for download at https://github.com/hpi-xnor
BibTeX
@inproceedings{yang2017bmxnet,
  author = {Haojin Yang and Martin Fritzsche and Christian Bartz and Christoph Meinel},
  title = {{BMXNet: An Open-Source Binary Neural Network Implementation Based on MXNet}},
  booktitle = {ACM Multimedia},
  year = {2017},
  url = {https://openreview.net/forum?id=wcAamzpzQ9},
}
Figure