Learning Channel-Wise Interactions for Binary Convolutional Neural Networks

Ziwei Wang, Jiwen Lu, Chenxin Tao, Jie Zhou, Qi Tian

Binary Neural Networks (BNNs) represent a significant frontier for deploying deep learning on resource-constrained edge devices. By quantizing weights and activations to 1-bit values, BNNs replace power-hungry floating-point operations with efficient bitwise XNOR and population count (bitcount) operations. However, this extreme compression often leads to a substantial degradation in predictive accuracy. Researchers from Tsinghua University and Huawei Noah’s Ark Lab have proposed CI-BCNN, a novel approach that leverages reinforcement learning to model and exploit channel-wise interactions, effectively mitigating the information loss inherent in binarization.

The Challenge of Sign Inconsistency

The primary hurdle in BNN performance is quantization error. Standard binarization techniques often fail to preserve the most critical information: the sign of the feature map values. In a full-precision network, a positive value indicates the presence of a feature. In a BNN, noise introduced during the XNOR-bitcount process can cause this sign to flip, leading to "inconsistent signs." This phenomenon results in significant information loss as errors propagate through the network layers.

Comparison of full-precision, standard binary (Xnor-Net), and CI-BCNN feature maps, highlighting the correction of inconsistent signs.

As illustrated in Figure 1, while traditional binary networks like Xnor-Net may produce incorrect signs (red circle), CI-BCNN utilizes channel-wise interactions to "correct" these flips (blue circle), bringing the binary feature maps closer to their full-precision counterparts.

CI-BCNN: Mining Channel Dependencies

The CI-BCNN architecture introduces two key components to address sign inconsistency: Interacted Bitcount and Channel-wise Interaction Mining.

Interacted Bitcount

Unlike standard layers where channels are processed independently, CI-BCNN allows channels to influence one another. A "teacher" channel can provide a prior—essentially a corrective offset—to a "student" channel. This interaction ensures that the student channel's output sign remains consistent with the underlying feature representation, even under heavy quantization.

Mining Interactions via Reinforcement Learning

To determine the optimal set of interactions without exhaustive manual tuning, the authors employ a reinforcement learning (RL) agent. This agent, structured as an Encoder-Decoder RNN, treats the channel interaction topology as a Directed Acyclic Graph (DAG).

The CI-BCNN framework using an RL-based encoder-decoder to mine and modify channel interaction graphs across layers.

As shown in Figure 4, the RL agent observes the state of the network and performs actions such as creating, deleting, or reassigning interaction edges. The agent is rewarded based on the reduction in prediction loss, effectively learning a "conversation" structure between channels that maximizes information flow.

Experimental Results

The CI-BCNN framework was evaluated on the CIFAR-10 and ImageNet (ILSVRC12) benchmarks using VGG and ResNet architectures.

  • ImageNet Performance: On ResNet-18, CI-BCNN achieved a state-of-the-art 59.9% top-1 accuracy, significantly outperforming Xnor-Net (51.2%) and Bi-Real-Net (56.4%).
  • CIFAR-10 Accuracy: Using a VGG-small backbone, the model reached 92.47% accuracy, nearly matching the 93.20% achieved by the full-precision baseline.
  • Efficiency: The model maintains the high efficiency of BNNs. Compared to a full-precision ResNet-18, CI-BCNN offers an 11.17x reduction in storage and an 11.75x reduction in computational complexity (FLOPs).

Conclusion

CI-BCNN demonstrates that the redundancy in convolutional channels can be transformed into a corrective mechanism for binarization errors. By using reinforcement learning to mine these hidden dependencies, the method achieves a superior balance between efficiency and accuracy, paving the way for more robust deep learning on the edge.

Summary updated

Abstract

In this paper, we propose a channel-wise interaction based binary convolutional neural network learning method (CI-BCNN) for efficient inference. Conventional methods apply xnor and bitcount operations in binary convolution with notable quantization error, which usually obtains inconsistent signs in binary feature maps compared with their full-precision counterpart and leads to significant information loss. In contrast, our CI-BCNN mines the channel-wise interactions, through which prior knowledge is provided to alleviate inconsistency of signs in binary feature maps and preserves the information of input samples during inference. Specifically, we mine the channel-wise interactions by a reinforcement learning model, and impose channel-wise priors on the intermediate feature maps through the interacted bitcount function. Extensive experiments on the CIFAR-10 and ImageNet datasets show that our method outperforms the state-of-the-art binary convolutional neural networks with less computational and storage cost.
BibTeX
@inproceedings{wang2019learning,
  author = {Ziwei Wang and Jiwen Lu and Chenxin Tao and Jie Zhou and Qi Tian},
  title = {{Learning Channel-Wise Interactions for Binary Convolutional Neural Networks}},
  booktitle = {IEEE/CVF Conference on Computer Vision and Pattern Recognition},
  year = {2019},
  doi = {10.1109/cvpr.2019.00066},
}
Figure