Feature Recalibration in Deep Learning via Depthwise Squeeze and Refinement Operations

Xing-peng Zhang, Xiaohong ZhangIEEE Access 2020 · DOI

Abstract

Feature recalibration is a critical mechanism for improving the representational power of neural networks by dynamically adjusting channel-wise feature responses. Traditional methods, such as Squeeze-and-Excitation (SE) blocks, rely on Global Average Pooling (GAP) and fully connected layers, which can lead to spatial information loss and undesirable channel dependencies. This paper introduces the Depthwise Squeeze and Refinement (DSR) block, which utilizes parameterized depthwise convolutions to capture fine-grained spatial patterns and maintain channel independence. Evaluated on ImageNet32 and CIFAR100, DSR-integrated architectures consistently outperform SE and CBAM baselines, offering a more precise and efficient approach to feature recalibration.

Introduction

In modern deep learning, the ability of a network to focus on informative features while suppressing noise is paramount. Feature recalibration modules have become standard components in state-of-the-art architectures. However, the widely adopted SE block suffers from two primary limitations:

  1. Spatial Information Loss: GAP reduces an entire feature map to a single scalar, ignoring the spatial distribution of features.
  2. Channel Dependency Disruption: The use of fully connected layers to learn inter-channel relationships can interfere with the specific characteristics of individual feature maps.

To address these challenges, we propose the DSR block, a lightweight unit that replaces heuristic pooling with learned spatial weighting.

Methodology: The DSR Block

The DSR block operates through two distinct stages: Depthwise Squeeze and Nonlinear Refinement.

Depthwise Squeeze

Unlike GAP, which treats all pixels equally, the DSR block employs a two-layer depthwise convolution. This operation allows the network to learn a unique spatial weighting for each channel independently. By processing each channel with its own set of filters, the model can identify and prioritize discriminative regions within each feature map without mixing information across channels.

Architecture of the Depthwise Squeeze and Refinement (DSR) block.

Nonlinear Refinement

Following the squeeze operation, a Sigmoid activation function is applied directly to the extracted global characteristics. This generates a scaling factor for each channel. By omitting the fully connected layers found in SE blocks, DSR ensures that the recalibration of a feature map is derived solely from its own learned spatial features, preserving channel independence.

Architectural Integration

The DSR block is designed as a "plug-and-play" module. It can be easily integrated into various architectures, such as ResNet and ShuffleNet-v2, to form "DSR-Nets." In residual architectures, the DSR mechanism is applied to the transformation branch before being summed with the identity shortcut.

Integration of the DSR mechanism into a residual block (DSE-residual block).

Experimental Evaluation

The efficacy of the DSR block was validated on two major image classification benchmarks: ImageNet32 and CIFAR100.

Key Results

  • ImageNet32: A DSR-ResNet50 achieved a top-1 accuracy of 58.22%, significantly surpassing the 55.87% achieved by the SE-ResNet50 baseline.
  • ShuffleNet-v2: On the ShuffleNet-v2-116 architecture, the DSR block provided a 3.83 percentage point improvement over the SE block, demonstrating its effectiveness in lightweight models.
  • Efficiency: Despite the addition of learned parameters, the DSR block remains computationally efficient. In some cases, such as the "ResNet50-new" configuration, the DSR-integrated model utilized fewer total parameters than the SE version while delivering superior performance.

Discussion and Conclusion

The DSR block represents a shift from heuristic feature squeezing to learned, channel-independent recalibration. By utilizing depthwise convolutions as "smart poolers," the model retains critical spatial information that is typically lost in global pooling operations. While the depthwise operations introduce a slight increase in FLOPs compared to parameterless pooling, the gains in accuracy and the preservation of channel independence justify the trade-off. Future work will explore the scalability of DSR to higher-resolution inputs and its application in multi-task learning environments.

Summary updated

BibTeX
@article{zhang2020feature,
  author = {Xing-peng Zhang and Xiaohong Zhang},
  title = {{Feature Recalibration in Deep Learning via Depthwise Squeeze and Refinement Operations}},
  journal = {IEEE Access},
  year = {2020},
  doi = {10.1109/access.2020.2990658},
}
Figure