HBONet: Harmonious Bottleneck on Two Orthogonal Dimensions.

Duo Li, Aojun Zhou, Anbang Yao

Introduction

In the domain of edge AI, computational efficiency is paramount. While architectures such as MobileNet and ShuffleNet have significantly advanced mobile vision through depthwise separable convolutions, they often encounter a performance plateau when operating under extremely restricted computational budgets (e.g., below 40 MFLOPs).

HBONet addresses this limitation by introducing the Harmonious Bottleneck (HBO). The authors identify a critical constraint in existing bottleneck designs: they primarily optimize the channel dimension while maintaining a constant spatial resolution within the block. HBONet proposes treating spatial and channel dimensions as orthogonal axes that can be jointly optimized to maximize representational capacity within a fixed FLOP budget.

Methods: The Harmonious Bottleneck

The core innovation of HBONet is the HBO module. Unlike the standard "Inverted Bottleneck" used in MobileNetV2, which expands and then contracts channels at a fixed resolution, the HBO module incorporates a nested spatial transformation.

The Harmonious Bottleneck (HBO) module architecture, illustrating the joint optimization of spatial and channel dimensions.

The module follows a four-step process:

  1. Spatial Contraction: A strided depthwise convolution downsamples the spatial resolution at the beginning of the block.
  2. Channel Expansion-Contraction: Within this reduced spatial footprint, the model performs channel expansion and contraction. The reduced resolution allows for a significantly wider channel expansion without exceeding the FLOP limit.
  3. Spatial Expansion: Upsampling restores the spatial resolution to align with the shortcut connection.
  4. Bilateral Symmetry & Feature Reuse: The module utilizes a concatenation path where half of the output channels are derived directly from the input, promoting feature reuse and further reducing computational overhead.

Key Insights

  • Spatial-Channel Reciprocity: Shrinking the spatial map provides the "computational budget" required to increase channel depth, leading to more diverse feature representations.
  • Micro-Architecture Optimization: By moving spatial downsampling inside the bottleneck block rather than treating it as a macro-architectural transition, HBONet achieves more granular control over model complexity.
  • Preserving Linearity: To prevent information loss in narrow layers, HBONet maintains linear bottlenecks, consistent with the design principles of MobileNetV2.

Experimental Results

HBONet was evaluated on ImageNet classification, PASCAL VOC object detection, and Market-1501 person re-identification. The results consistently demonstrate that HBONet outperforms MobileNetV2, particularly at the lower boundary of computational resources.

Comparison of Top-1 Accuracy vs. MFLOPs on ImageNet. HBONet shows significant gains over MobileNetV2, especially in ultra-lightweight regimes.

On ImageNet, HBONet achieved a 6.6% top-1 accuracy improvement over MobileNetV2 at the ~14 MFLOPs level. In object detection tasks, HBONet-based SSD models showed up to a 6.3% mAP improvement in constrained settings. Furthermore, for person re-identification, the architecture provided a 5.0% Rank-1 accuracy boost.

Conclusion and Future Work

HBONet demonstrates that joint optimization of spatial and channel dimensions can push the boundaries of ultra-lightweight neural networks. While the current design is hand-engineered, future work could involve integrating the HBO module into Neural Architecture Search (NAS) frameworks to discover even more efficient configurations. Additionally, exploring the impact of hardware-specific optimizations for upsampling operations remains a key area for practical deployment.

Summary updated

Abstract

MobileNets, a class of top-performing convolutional neural network architectures in terms of accuracy and efficiency trade-off, are increasingly used in many resource-aware vision applications. In this paper, we present Harmonious Bottleneck on two Orthogonal dimensions (HBO), a novel architecture unit, specially tailored to boost the accuracy of extremely lightweight MobileNets at the level of less than 40 MFLOPs. Unlike existing bottleneck designs that mainly focus on exploring the interdependencies among the channels of either groupwise or depthwise convolutional features, our HBO improves bottleneck representation while maintaining similar complexity via jointly encoding the feature interdependencies across both spatial and channel dimensions. It has two reciprocal components, namely spatial contraction-expansion and channel expansion-contraction, nested in a bilaterally symmetric structure. The combination of two interdependent transformations performing on orthogonal dimensions of feature maps enhances the representation and generalization ability of our proposed module, guaranteeing compelling performance with limited computational resource and power. By replacing the original bottlenecks in MobileNetV2 backbone with HBO modules, we construct HBONets which are evaluated on ImageNet classification, PASCAL VOC object detection and Market-1501 person re-identification. Extensive experiments show that with the severe constraint of computational budget our models outperform MobileNetV2 counterparts by remarkable margins of at most 6.6%, 6.3% and 5.0% on the above benchmarks respectively. Code and pretrained models are available at https://github.com/d-li14/HBONet.
BibTeX
@inproceedings{li2019hbonet,
  author = {Duo Li and Aojun Zhou and Anbang Yao},
  title = {{HBONet: Harmonious Bottleneck on Two Orthogonal Dimensions.}},
  booktitle = {ICCV},
  year = {2019},
  doi = {10.1109/iccv.2019.00341},
}
Figure