Introduction
Deep learning architectures for image classification have long grappled with the balance between feature exploration and feature reuse. ResNet popularized residual connections to facilitate feature reuse and gradient flow, while DenseNet introduced dense connectivity to maximize feature exploration. However, DenseNet's quadratic growth in connections leads to significant parameter redundancy. This paper introduces HCGNet (Hybrid Connectivity Gated Network), a novel architecture that harmonizes these two paradigms through a gated hybrid connectivity pattern.
Hybrid Connectivity Architecture
The fundamental building block of HCGNet is the Hybrid Block. Unlike standard DenseNets that rely solely on concatenation, HCGNet employs a nested aggregation strategy. Globally, modules within a block are connected via dense concatenation (dense connectivity), ensuring robust feature exploration. Locally, within each module, residual additions (residual connectivity) are used to maintain efficient feature flow and reuse.
This hybrid approach allows the network to capture the representational power of dense connections while mitigating the explosion of redundant parameters. By structuring the network this way, HCGNet achieves superior performance with significantly fewer modules per block compared to traditional DenseNets.
The SMG Module: Squeeze, Multi-scale, and Gated
At the heart of the Hybrid Block is the SMG Module, which consists of three primary components designed for efficient feature extraction and management:
- Squeeze Cell: This component utilizes 1x1 and 3x3 group convolutions to condense incoming feature maps, reducing computational overhead.
- Multi-scale Excitation Cell: To capture diverse spatial information, this cell employs parallel 3x3 and 5x5 depthwise convolutions. This "wider" local structure allows the model to learn features at multiple scales without the depth-related parameter cost.
- Gated Mechanisms: The module incorporates an Update Gate and a Forget Gate. The Update Gate uses spatial and channel attention to intelligently fuse multi-scale features. Conversely, the Forget Gate selectively decays redundant information from the residual path, mimicking biological neural filtering.
Notably, the authors utilize the tanh activation function within these gates. They argue that tanh is better suited for preserving information in low-dimensional spaces, and the hybrid connectivity effectively prevents the vanishing gradient issues typically associated with non-ReLU activations.
Experimental Evaluation
HCGNet was rigorously tested across multiple benchmarks, demonstrating state-of-the-art efficiency:
- CIFAR-10/100: HCGNet-A3 outperformed NASNet-A while utilizing 4.5x fewer parameters, highlighting its architectural efficiency.
- ImageNet: HCGNet-C achieved a 19.5% Top-1 error rate with 42.2M parameters. In comparison, ResNet-101 requires 44.6M parameters to reach a significantly higher error rate of 23.4%.
- Transfer Learning: When integrated into the Mask R-CNN framework for MS-COCO, HCGNet backbones consistently outperformed ResNet-50 in both object detection and instance segmentation tasks.
Robustness and Interpretability
Beyond raw accuracy, HCGNet exhibits enhanced adversarial robustness. In tests against FGSM attacks, it maintained higher accuracy than both ResNet and DenseNet, likely due to the smoother information flow provided by the gated hybrid connections. Furthermore, network dissection analysis revealed that HCGNet learns more distinct and high-level visual detectors (e.g., specific textures and object parts) than traditional backbones, suggesting a more meaningful internal representation of visual data.
Conclusion
HCGNet represents a significant step forward in efficient CNN design. By combining dense and residual connectivity through intelligent gating mechanisms, it effectively addresses the redundancy issues of DenseNet while surpassing the representational limits of ResNet. Its performance across classification, detection, and segmentation tasks makes it a highly versatile backbone for modern computer vision.
Summary updated