Deeply Supervised Rotation Equivariant Network for Lesion Segmentation in Dermoscopy Images

Xiaomeng Li, Lequan Yu, Chi-Wing Fu, Pheng-Ann Heng

Abstract

Automatic lesion segmentation in dermoscopy images is an essential step for computer-aided diagnosis of melanoma. The dermoscopy images exhibits rotational and reflectional symmetry, however, this geometric property has not been encoded in the state-of-the-art convolutional neural networks based skin lesion segmentation methods. In this paper, we present a deeply supervised rotation equivariant network for skin lesion segmentation by extending the recent group rotation equivariant network. Specifically, we propose the G-upsampling and G-projection operations to adapt the rotation equivariant classification network for our skin lesion segmentation problem. To further increase the performance, we integrate the deep supervision scheme into our proposed rotation equivariant segmentation architecture. The whole framework is equivariant to input transformations, including rotation and reflection, which improves the network efficiency and thus contributes to the segmentation performance. We extensively evaluate our method on the ISIC 2017 skin lesion challenge dataset. The experimental results show that our rotation equivariant networks consistently excel the regular counterparts with the same model complexity under different experimental settings. Our best model achieves 77.23%(JA) on the test dataset, outperforming the state-of-the-art challenging methods and further demonstrating the effectiveness of our proposed deeply supervised rotation equivariant segmentation network.

Introduction

In medical imaging, particularly dermoscopy (close-up skin photography), the orientation of the camera does not change the underlying pathology. A melanoma remains a melanoma regardless of whether the image is rotated or flipped. While standard Convolutional Neural Networks (CNNs) are inherently translation equivariant—meaning a shift in the input results in a corresponding shift in the feature map—they are not naturally rotation equivariant.

Standard convolutions are translation equivariant but fail to maintain equivariance under rotation.

To compensate for this lack of geometric awareness, practitioners typically rely on heavy data augmentation, feeding the network numerous rotated versions of the same image. However, this approach is computationally inefficient, as the network must dedicate its internal parameters to learning the same features at multiple orientations. This research addresses the need for a more mathematically elegant and efficient method to encode these symmetries directly into the network architecture for image segmentation.

Methodology

The proposed architecture is a Deeply Supervised Rotation Equivariant Network that adapts Group Equivariant CNNs (G-CNNs) for pixel-level segmentation tasks.

G-Convolutions and Symmetry Groups

Instead of standard filters, the model utilizes G-convolutions that operate over a "group" of transformations. Specifically, it uses the $p4m$ group, which includes 90-degree rotations and reflections. This design allows for significantly higher weight sharing; by sharing weights across eight different orientations, the network increases its expressive capacity without increasing the total number of parameters.

Novel G-Layers for Segmentation

To transition from low-resolution feature maps back to a full-sized segmentation mask, the authors introduced two specialized layers:

  • G-upsampling: This layer increases spatial resolution while preserving the rotation-aware data structure.
  • G-projection: This operation "flattens" the rotation dimensions (typically via average pooling) to produce a standard 2D score map suitable for final segmentation.

Architecture and Deep Supervision

The model employs a ResNet34 backbone with U-Net-style skip connections. To stabilize training and enhance performance, the authors integrated a Deep Supervision scheme. This forces the model to produce and refine segmentation predictions at three different spatial scales simultaneously, ensuring robust feature learning throughout the network.

The complete deeply supervised rotation equivariant network architecture, illustrating the flow from input to multi-scale G-projections.

Experimental Results

The model was evaluated using the ISIC 2017 skin lesion challenge dataset, consisting of 2,000 training images, 150 validation images, and 600 test images. All images were resized to $224 \times 224$ for consistency.

The proposed rotation-equivariant model achieved state-of-the-art results on the benchmark:

  • Jaccard Index (JA): 0.772 (surpassing the previous state-of-the-art of 0.765).
  • Dice Coefficient (DI): 0.856.

A direct comparison between a standard ResNetFCN34 and the rotation-equivariant version (maintaining identical model complexity) showed a significant jump in the Jaccard Index from 71.27% to 74.54%. This demonstrates that encoding geometric priors is a powerful way to improve performance, especially in specialized medical datasets where data is relatively scarce.

Conclusion

By building rotational and reflectional symmetry directly into the network's design, the researchers have developed a more efficient and accurate model for skin lesion segmentation. While currently limited to discrete 90-degree rotations, the framework provides a strong foundation for future work involving continuous rotations and scaling equivariance, further bridging the gap between clinical photography and robust automated diagnosis.

Summary updated

Abstract

Automatic lesion segmentation in dermoscopy images is an essential step for computer-aided diagnosis of melanoma. The dermoscopy images exhibits rotational and reflectional symmetry, however, this geometric property has not been encoded in the state-of-the-art convolutional neural networks based skin lesion segmentation methods. In this paper, we present a deeply supervised rotation equivariant network for skin lesion segmentation by extending the recent group rotation equivariant network~\cite{cohen2016group}. Specifically, we propose the G-upsampling and G-projection operations to adapt the rotation equivariant classification network for our skin lesion segmentation problem. To further increase the performance, we integrate the deep supervision scheme into our proposed rotation equivariant segmentation architecture. The whole framework is equivariant to input transformations, including rotation and reflection, which improves the network efficiency and thus contributes to the segmentation performance. We extensively evaluate our method on the ISIC 2017 skin lesion challenge dataset. The experimental results show that our rotation equivariant networks consistently excel the regular counterparts with the same model complexity under different experimental settings. Our best model achieves 77.23\%(JA) on the test dataset, outperforming the state-of-the-art challenging methods and further demonstrating the effectiveness of our proposed deeply supervised rotation equivariant segmentation network. Our best model also outperforms the state-of-the-art challenging methods, which further demonstrate the effectiveness of our proposed deeply supervised rotation equivariant segmentation network.
BibTeX
@inproceedings{li2018deeply,
  author = {Xiaomeng Li and Lequan Yu and Chi-Wing Fu and Pheng-Ann Heng},
  title = {{Deeply Supervised Rotation Equivariant Network for Lesion Segmentation in Dermoscopy Images}},
  booktitle = {OR 2.0/CARE/CLIP/ISIC @ MICCAI},
  year = {2018},
  doi = {10.1007/978-3-030-01201-4\_25},
}
Figure