'Project & Excite' Modules for Segmentation of Volumetric Medical Scans

Anne-Marie Rickmann, Abhijit Guha Roy, Ignacio Sarasua, Nassir Navab, Christian Wachinger

Introduction: The 3D Dilemma in Medical Imaging

Deep learning has revolutionized medical imaging, yet processing volumetric scans (MRI, CT, PET) presents a unique challenge. While 3D Fully Convolutional Neural Networks (F-CNNs) can capture spatial context across slices, they are computationally intensive and prone to overfitting, especially given the typically small size of annotated medical datasets.

To address efficiency, researchers previously adapted "Squeeze and Excitation" (SE) blocks. However, standard 3D SE blocks "squeeze" entire volumes into single scalars per channel, destroying critical spatial information. In medical segmentation, the spatial location of a feature is often as vital as the feature itself. The 'Project & Excite' (PE) module is proposed as a lightweight architectural plugin that preserves this spatial distribution while maintaining parameter efficiency.

Methods: The 'Project & Excite' Architecture

The PE module is designed to recalibrate 3D feature maps ($H \times W \times D \times C$) by considering spatial dependencies along principal axes.

The process consists of three primary stages:

  1. Projection: Instead of global averaging, the module performs three separate average pooling operations along the $H$, $W$, and $D$ axes. This generates three projection vectors that summarize the spatial distribution of features along each dimension.
  2. Excitation: These projections are tiled back to the original spatial dimensions and summed. This combined tensor passes through two $1 \times 1 \times 1$ convolutional layers (with ReLU and Sigmoid activations), allowing the network to learn complex inter-dependencies between channels within their spatial context.
  3. Recalibration: The learned weights are used to multiply the original feature map, effectively "exciting" relevant regions and suppressing noise.

In practice, these PE blocks are integrated into a 3D U-Net, placed immediately after each encoder and decoder block to refine features at multiple scales.

Experimental Setup and Datasets

The method was evaluated on two benchmarks characterized by limited data and severe class imbalance:

  • MALC (Multi-Atlas Labelling Challenge): 30 T1 MRI brain volumes for segmenting 32 structures.
  • Visceral Dataset: 20 contrast-enhanced CT scans for segmenting 14 organs in the thorax and abdomen.

Key Results and Discussion

The PE module demonstrated significant improvements over both baseline 3D U-Nets and standard 3D channel-SE blocks. On the MALC dataset, the PE-equipped model achieved a mean Dice score of 0.843, a substantial jump from the baseline's 0.802.

Qualitative comparison of segmentation results across different models for brain MRI and lung CT scans.

A key insight is that the PE module acts as a "small structure specialist." While gains for large organs like the liver were modest, small or thin structures like the amygdala or trachea saw performance boosts of up to 36%. Visual analysis confirms that the PE model accurately identifies structures—such as the bifurcation of the trachea—that baseline models often miss entirely.

Crucially, these gains come with minimal overhead. While adding more convolutional layers typically increases parameters by ~40%, the PE module achieves superior results with only a ~2% increase in model complexity.

Conclusion

The 'Project & Excite' module offers a powerful, "plug-and-play" solution for enhancing 3D medical image segmentation. By replacing aggressive global pooling with axis-wise projections, it maintains a global receptive field without the risk of overfitting common in deeper 3D architectures. Future work may explore the integration of PE modules into other volumetric tasks such as 3D object detection or classification.

Summary updated

BibTeX
@inproceedings{rickmann2019project,
  author = {Anne-Marie Rickmann and Abhijit Guha Roy and Ignacio Sarasua and Nassir Navab and Christian Wachinger},
  title = {{'Project \& Excite' Modules for Segmentation of Volumetric Medical Scans}},
  booktitle = {International Conference on Medical Image Computing and Computer-Assisted Intervention},
  year = {2019},
  doi = {10.1007/978-3-030-32245-8\_5},
}
Figure