Recalibrating Fully Convolutional Networks With Spatial and Channel "Squeeze and Excitation" Blocks

Abhijit Guha Roy, Nassir Navab, Christian Wachinger

Abstract

This paper introduces a novel architectural enhancement for medical image segmentation by integrating "Squeeze & Excitation" (SE) blocks into Fully Convolutional Neural Networks (F-CNNs). By adaptively recalibrating feature maps along both channel and spatial dimensions, the proposed Concurrent Spatial and Channel SE (scSE) block significantly improves segmentation accuracy. Evaluations on three diverse medical imaging datasets (MRI, CT, and OCT) demonstrate that adding scSE blocks to standard architectures like U-Net yields a 4-9% boost in Dice scores with a negligible 1.5% increase in model parameters, outperforming simple increases in network depth.

Introduction

In the domain of semantic segmentation, Fully Convolutional Neural Networks (F-CNNs) such as U-Net are the standard. However, standard convolutions treat all channels and spatial pixels with equal importance within a local neighborhood. They lack an inherent mechanism to "recalibrate" themselves to ignore noise or highlight specific anatomical structures that may be small or irregularly shaped. This research addresses whether networks can learn to identify which features are task-relevant and suppress the rest through adaptive recalibration.

Methodology

The authors build upon the original "Squeeze & Excitation" (SE) concept—initially designed for classification—and adapt it for the pixel-level requirements of segmentation. Three variants are proposed:

  1. Channel SE (cSE): This block "squeezes" spatial information into a single vector using global average pooling and then "excites" (reweights) the channels. This provides a global view of the image to determine which features are globally important (the "what").
  2. Spatial SE (sSE): A novel contribution that "squeezes" channel information using a $1 \times 1$ convolution to create a single-channel spatial map. This acts as an attention mechanism, identifying where the network should focus spatially (the "where").
  3. Concurrent Spatial and Channel SE (scSE): This combines both blocks. Among various merging strategies, Max-Out—taking the maximum value between the two blocks at every pixel—was found to be most effective, as it encourages competition between spatial and channel importance.
Architectural diagrams of (a) cSE, (b) sSE, and (c) scSE blocks.

These blocks are designed to be "plug-and-play," allowing integration after encoder and decoder blocks in standard architectures like U-Net, SD-Net, and FC-DenseNet.

Experimental Setup

The approach was validated on three challenging medical imaging datasets:

  • MALC (Brain MRI): Segmenting 27 cortical and sub-cortical structures, characterized by severe class imbalance.
  • Visceral (Whole-body CT): Segmenting 10 abdominal organs with highly variable shapes.
  • Duke (Retinal OCT): Segmenting 7 thin retinal layers (5-6 pixels wide) amidst significant noise.
Various placement strategies (P1-P6) for integrating SE blocks within an F-CNN architecture.

The researchers explored several placement strategies (P1-P6), ultimately finding that placing blocks throughout the encoder and decoder paths (P5/P6) yielded the best results.

Key Results and Discussion

  • Performance Gains: On the MALC dataset, the Dice score for U-Net improved from 0.763 to 0.851, representing an ~8.8% absolute improvement. This consistency was observed across all tested architectures and modalities.
  • Efficiency vs. Depth: The scSE block proved far more efficient than increasing network depth. For instance, adding an extra encoder/decoder block to SD-Net increased complexity by 33% for a 5% gain, whereas scSE provided a 9% gain for only 1.5% more parameters.
  • Small Structure Recovery: Qualitatively, SE blocks enabled the recovery of tiny structures (e.g., the amygdala) that baseline models often missed. It also reduced "over-segmentation" where the model would erroneously bleed into adjacent tissues.
  • Competitive Aggregation: The superiority of the Max-Out strategy suggests that letting spatial and channel excitations compete for dominance leads to more discriminative features than simple addition.

Conclusion

The integration of spatial and channel SE blocks provides a powerful, parameter-efficient method for boosting F-CNN performance in medical imaging. While highly effective, future work may explore the sensitivity of the bottleneck ratio ($r$) and the extension of these concurrent blocks to fully 3D volumetric data.

Summary updated

BibTeX
@article{roy2019recalibrating,
  author = {Abhijit Guha Roy and Nassir Navab and Christian Wachinger},
  title = {{Recalibrating Fully Convolutional Networks With Spatial and Channel "Squeeze and Excitation" Blocks}},
  journal = {IEEE Trans. Medical Imaging},
  year = {2019},
  doi = {10.1109/tmi.2018.2867261},
}

Loading related content...

Figure