Prototype Learning Guided Hybrid Network for Breast Tumor Segmentation in DCE-MRI

Lei Zhou, Yuzhong Zhang, Jiadong Zhang, Xuejun Qian, Chen Gong, Kun Sun, Zhongxiang Ding, Xing Wang, Zhenhui Li, Zaiyi Liu, Dinggang ShenIEEE TMI 2024 · DOI · OpenReview

Abstract

Automated breast tumor segmentation using dynamic contrast-enhanced magnetic resonance imaging (DCE-MRI) is vital for clinical practice. However, achieving high accuracy often requires computationally expensive models. We propose the Prototype Learning Guided Hybrid Network (PLHN), which combines convolutional neural networks (CNNs) and Transformer layers to balance computational efficiency and segmentation performance. PLHN features a dual-encoder architecture and a prototype learning module that uses online clustering to enhance discriminative capability. Experimental results on private and public datasets show that PLHN outperforms state-of-the-art (SOTA) methods while maintaining low computational costs. Furthermore, the automated masks generated by PLHN effectively support downstream clinical tasks, such as identifying HER2-positive subtypes.

Introduction

Dynamic contrast-enhanced magnetic resonance imaging (DCE-MRI) is a cornerstone of breast cancer detection. Despite its importance, manual tumor segmentation is a labor-intensive process prone to variability. While deep learning has significantly automated this task, current models face two primary challenges:

  1. Complexity-Performance Trade-off: High-accuracy models typically demand massive computational resources, hindering clinical deployment.
  2. Class Imbalance: Tumor voxels represent only a small fraction of the total breast volume, making it difficult for standard networks to distinguish tumors from surrounding background tissue.

PLHN addresses these issues by integrating global context modeling with local feature extraction in a computationally lean architecture.

Methods: The PLHN Architecture

The PLHN architecture is designed to capture both fine-grained textures and long-range spatial dependencies without the overhead of traditional hybrid models.

Architecture of the Prototype Learning Guided Hybrid Network (PLHN), featuring dual encoders and a prototype-guided prediction module.

Hybrid Dual-Encoder Design

Unlike standard U-Net variants, PLHN employs two parallel encoder subnetworks. Subnetwork-1 extracts local features for skip connections, while Subnetwork-2 focuses on bottleneck features for a 3D Transformer layer (based on the Swin-Transformer). This separation allows the model to process high-resolution images efficiently by avoiding the direct feeding of large volumes into the Transformer layers.

Prototype Learning Guided Prediction

The core innovation of PLHN is the Prototype Learning module. Instead of relying solely on a final classification layer, the network maintains representative feature vectors (prototypes) for tumor and background classes. These prototypes are updated via online clustering during training. A Query-Key-Value (QKV) attention module then fuses the decoder's features with these prototypes, significantly improving the model's ability to discriminate between similar-looking tissues.

Two-Stage Optimization

To stabilize training, the authors utilize a staged approach: the CNN backbone is optimized first, followed by a second stage that fine-tunes the entire hybrid system, including the Transformer and prototype modules.

Experimental Evaluation

Datasets

The model was validated across diverse datasets to ensure generalizability:

  • Internal Dataset: 1,065 DCE-MRI cases from Ruijin and Guangdong Provincial hospitals.
  • External Dataset: 100 public cases from Yunnan Cancer Hospital.
  • Cross-Domain Benchmarks: Testing on BraTS (Brain Tumor) and Thymoma (CT scan) datasets to prove architectural versatility.

Key Results

  • Segmentation Performance: PLHN achieved a Dice Similarity Coefficient (DSC) of 80.6% on the internal test set and 78.1% on the external set, outperforming 13 SOTA methods, including nnU-Net and UNETR++.
  • Efficiency: The model is remarkably lightweight, requiring only 11.0M parameters and 146.1G FLOPs. It is approximately 17 times faster than the previous SOTA model (ALMN).
  • Statistical Significance: Improvements over leading baselines were confirmed to be statistically significant ($p < 0.05$).

Clinical Utility: HER2 Status Prediction

A critical finding of this research is the model's downstream utility. Features extracted from PLHN's automated masks were used to predict HER2 status, a vital biomarker for treatment planning. The automated masks achieved accuracy comparable to expert-drawn manual masks, demonstrating that PLHN can reliably replace manual segmentation in clinical workflows.

Conclusion

PLHN represents a significant step forward in medical image segmentation by successfully merging the strengths of CNNs and Transformers. By utilizing parallel encoders and prototype learning, the network achieves superior accuracy with a fraction of the computational cost of its predecessors. Future work will focus on reducing false positives in tissues that mimic tumor enhancement and incorporating multi-phase temporal data to further refine diagnostic accuracy.

Summary updated

Abstract

Automated breast tumor segmentation on the basis of dynamic contrast-enhancement magnetic resonance imaging (DCE-MRI) has shown great promise in clinical practice, particularly for identifying the presence of breast disease. However, accurate segmentation of breast tumor is a challenging task, often necessitating the development of complex networks. To strike an optimal trade-off between computational costs and segmentation performance, we propose a hybrid network via the combination of convolution neural network (CNN) and transformer layers. Specifically, the hybrid network consists of a encoder-decoder architecture by stacking convolution and decovolution layers. Effective 3D transformer layers are then implemented after the encoder subnetworks, to capture global dependencies between the bottleneck features. To improve the efficiency of hybrid network, two parallel encoder subnetworks are designed for the decoder and the transformer layers, respectively. To further enhance the discriminative capability of hybrid network, a prototype learning guided prediction module is proposed, where the category-specified prototypical features are calculated through on-line clustering. All learned prototypical features are finally combined with the features from decoder for tumor mask prediction. The experimental results on private and public DCE-MRI datasets demonstrate that the proposed hybrid network achieves superior performance than the state-of-the-art (SOTA) methods, while maintaining balance between segmentation accuracy and computation cost. Moreover, we demonstrate that automatically generated tumor masks can be effectively applied to identify HER2-positive subtype from HER2-negative subtype with the similar accuracy to the analysis based on manual tumor segmentation. The source code is available at https://github.com/ZhouL-lab/PLHN.
BibTeX
@article{zhou2024prototype,
  author = {Lei Zhou and Yuzhong Zhang and Jiadong Zhang and Xuejun Qian and Chen Gong and Kun Sun and Zhongxiang Ding and Xing Wang and Zhenhui Li and Zaiyi Liu and Dinggang Shen},
  title = {{Prototype Learning Guided Hybrid Network for Breast Tumor Segmentation in DCE-MRI}},
  journal = {IEEE Transactions on Medical Imaging},
  year = {2024},
  doi = {10.1109/tmi.2024.3435450},
}
Figure