Abstract
HCGNet (Hybrid Convolutional Gated Network) addresses the architectural redundancy found in modern deep learning models by integrating the global feature exploration of DenseNet with the local efficiency of ResNet. Central to this architecture is the Squeeze Multi-scale Gated (SMG) module, which utilizes "Forget" and "Update" gates to actively manage feature flow and prevent representation collapse. Evaluations on CIFAR and ImageNet benchmarks show that HCGNet achieves superior performance compared to models like NASNet and ResNeXt while utilizing significantly fewer parameters. Furthermore, HCGNet demonstrates enhanced robustness to adversarial attacks and improved feature interpretability.
Introduction
Deep learning has been dominated by two primary architectural paradigms: ResNet, which employs additive residual connections for efficient feature reuse, and DenseNet, which uses concatenations to explore new feature combinations. While DenseNet is powerful, its quadratic growth in connections leads to significant redundancy and "superfluous features." Conversely, ResNet can suffer from "representation collapse" when excessive feature addition dilutes information. HCGNet seeks a "Goldilocks" zone, combining the strengths of both while mitigating their respective weaknesses through active feature management.
Architectural Design: Hybrid Connectivity and the SMG Module
The HCGNet architecture is built upon two foundational concepts: Hybrid Connectivity and the Squeeze Multi-scale Gated (SMG) module.
Hybrid Connectivity
HCGNet employs a dual-connectivity strategy. It utilizes global dense connectivity (concatenation) to link different modules, ensuring that early-stage features remain accessible throughout the network. Within each module, however, it applies local residual connectivity (addition) to refine features efficiently without the memory overhead of continuous concatenation.
The SMG Module
The SMG module is the primary processing unit of HCGNet, designed to reduce redundancy and enhance feature diversity. It consists of two main cells and two gates:
- Squeeze Cell: Compresses dense features using 1x1 and 3x3 group convolutions.
- Multi-scale Excitation Cell: Processes features through parallel 3x3 and 5x5 depthwise convolutions to capture multi-scale information simultaneously.
- Update Gate: An attention mechanism using
tanhactivations to dynamically fuse multi-scale features based on spatial and channel importance. - Forget Gate: A gating mechanism on the residual shortcut that learns to suppress redundant or noisy historical features before they are combined with new information, acting similarly to a biological synapse.
Experimental Results
HCGNet was evaluated across several standard benchmarks, including CIFAR-10/100, ImageNet (ILSVRC 2012), and MS-COCO for transfer learning.
- Efficiency and Performance: On CIFAR, HCGNet-A3 outperformed the auto-searched NASNet-A with 4.5x fewer parameters. On ImageNet, HCGNet-C achieved a Top-1 error of 19.5%, surpassing larger models like ResNeXt-101 and DPN-98.
- Redundancy Reduction: HCGNet-B matched or exceeded the accuracy of DenseNet-169 while reducing the number of stacked modules by 88%.
- Robustness: The gated architecture provides inherent resistance to adversarial attacks (e.g., FGSM) compared to standard ResNet and DenseNet models, likely due to its active feature filtering.
- Transferability: As a backbone for Mask R-CNN on MS-COCO, HCGNet-B demonstrated superior performance in object detection and instance segmentation tasks over ResNet-50 and AOGNet.
Key Insights: Active Feature Management and Interpretability
HCGNet shifts the architectural paradigm from passive feature stacking to active feature management. Network Dissection analysis reveals that HCGNet naturally learns a higher number of unique, high-level semantic detectors (e.g., specific object parts and textures) than its predecessors. This suggests that its internal representations are cleaner and more diverse, avoiding the redundancy common in purely dense architectures.
Conclusion
HCGNet demonstrates that architectural redundancy is not a prerequisite for high performance. By intelligently combining dense and residual connectivity with active gating mechanisms, HCGNet provides a highly efficient, robust, and interpretable framework for image classification and beyond. Future work may involve integrating these manual design principles into Neural Architecture Search (NAS) to discover even more optimized configurations.
Summary updated