TSESNet: Temporal-Spatial Enhanced Breast Tumor Segmentation in DCE-MRI Using Feature Perception and Separability

Jiezhou He, Xue Zhao, Zhiming Luo, Songzhi Su, Shaozi Li, Guojun ZhangIJCAI 2024 · OpenReview

Abstract

Accurate segmentation of breast tumors in dynamic contrast-enhanced magnetic resonance images (DCE-MRI) is critical for early diagnosis of breast cancer. However, this task remains challenging due to the wide range of tumor sizes, shapes, and appearances. Additionally, the complexity is further compounded by the high dimensionality and ill-posed artifacts present in DCE-MRI data. Furthermore, accurately modeling features in DCE-MRI sequences presents a challenge that hinders the effective representation of essential tumor characteristics. Therefore, this paper introduces a novel Temporal-Spatial Enhanced Network (TSESNet) for breast tumor segmentation in DCE-MRI. TSESNet leverages the spatial and temporal dependencies of DCE-MRI to provide a comprehensive representation of tumor features. To address sequence modeling challenges, we propose a Temporal-Spatial Contrastive Loss (TSCLoss) that maximizes the distance between different classes and minimizes the distance within the same class, thereby improving the separation between tumors and the background. Moreover, we design a novel Temporal Series Feature Fusion (TSFF) module that effectively integrates temporal MRI features from multiple time points, enhancing the model's ability to handle temporal sequences and improving overall performance. Finally, we introduce a simple and effective Tumor-Aware (TA) module that enriches feature representation to accommodate tumors of various sizes. We conducted comprehensive experiments to validate the proposed method and demonstrate its superior performance compared to recent state-of-the-art segmentation methods on two breast cancer DCE-MRI datasets.

1. Introduction: The Challenge of 4D Breast MRI

Breast cancer diagnosis relies heavily on Dynamic Contrast-Enhanced MRI (DCE-MRI), a modality that captures a sequence of 3D volumes over time to monitor the uptake and washout of contrast agents. This "4D" data provides vital physiological information but presents significant computational and modeling challenges. Tumors exhibit high variance in scale and appearance, and the optimal time point for visualization varies significantly between patients.

Overview of DCE-MRI temporal sequences and segmentation examples

Existing segmentation methods often struggle with the high dimensionality of DCE-MRI or rely on simplified representations like Maximum Intensity Projections, which discard valuable temporal nuances. TSESNet addresses these limitations by explicitly modeling temporal-spatial dependencies and enhancing feature separability through a weight-shared encoder architecture.

2. Methodology: TSESNet Architecture

The TSESNet framework incorporates three primary innovations designed to capture the unique characteristics of breast tumors in dynamic sequences.

Tumor-Aware (TA) Module

To handle tumors of all sizes, the TA module utilizes multi-scale atrous (dilated) convolutions. By employing different kernel sizes, the network can simultaneously capture fine-grained details for small nodules and a broader field of view for large masses. An integrated attention mechanism then fuses these multi-scale features to optimize representation.

Temporal Series Feature Fusion (TSFF)

Radiologists typically identify tumors by comparing post-contrast images to a baseline pre-contrast volume ($V_0$). The TSFF module automates this by using attention mechanisms to learn the relationships between $V_0$ and each subsequent volume ($V_1$ to $V_T$). This allows the model to dynamically weigh the most informative time points for the final segmentation.

Close-up of tumor enhancement in a subtraction slice

Temporal-Spatial Contrastive Loss (TSCLoss)

A key innovation is the TSCLoss, which moves beyond simple pixel-wise error minimization. It explicitly forces the model to cluster features of the same class (tumor) together while pushing different classes (tumor vs. healthy tissue) apart in the feature space. This contrastive approach is applied across both spatial and temporal dimensions, ensuring robust segmentation even as tumor appearance changes during the contrast cycle.

3. Experimental Results and Efficiency

TSESNet was evaluated on two datasets: the public BTS dataset (100 cases) and the private ST177 dataset (177 cases). The model demonstrated superior performance across all major metrics compared to seven state-of-the-art methods, including both 3D and 4D approaches.

  • Accuracy: TSESNet achieved a Dice score of 88.37% on ST177 and 78.95% on BTS, outperforming previous benchmarks by several percentage points.
  • Boundary Precision: The 95% Hausdorff Distance (HD95) on ST177 was reduced to 7.154 mm, nearly halving the error of some competing models.
  • Computational Efficiency: Despite its performance, TSESNet is highly efficient. It utilizes a weight-shared encoder to reduce parameters (~103M) and requires only 98.66G FLOPs, significantly less than Transformer-based models like SwinHR (314.72G FLOPs).

4. Conclusion

TSESNet represents a significant advancement in 4D medical image segmentation. By combining multi-scale spatial perception with sophisticated temporal fusion and contrastive learning, it achieves state-of-the-art accuracy while remaining computationally feasible for clinical environments. Future work may extend this temporal-spatial contrastive approach to other dynamic imaging modalities, such as cardiac or functional brain MRI.

Summary updated

Abstract

Accurate segmentation of breast tumors in dynamic contrast-enhanced magnetic resonance images (DCE-MRI) is critical for early diagnosis of breast cancer. However, this task remains challenging due to the wide range of tumor sizes, shapes, and appearances. Additionally, the complexity is further compounded by the high dimensionality and ill-posed artifacts present in DCE-MRI data. Furthermore, accurately modeling features in DCE-MRI sequences presents a challenge that hinders the effective representation of essential tumor characteristics. Therefore, this paper introduces a novel Temporal-Spatial Enhanced Network (TSESNet) for breast tumor segmentation in DCE-MRI. TSESNet leverages the spatial and temporal dependencies of DCE-MRI to provide a comprehensive representation of tumor features. To address sequence modeling challenges, we propose a Temporal-Spatial Contrastive Loss (TSCLoss) that maximizes the distance between different classes and minimizes the distance within the same class, thereby improving the separation between tumors and the background. Moreover, we design a novel Temporal Series Feature Fusion (TSFF) module that effectively integrates temporal MRI features from multiple time points, enhancing the model's ability to handle temporal sequences and improving overall performance. Finally, we introduce a simple and effective Tumor-Aware (TA) module that enriches feature representation to accommodate tumors of various sizes. We conducted comprehensive experiments to validate the proposed method and demonstrate its superior performance compared to recent state-of-the-art segmentation methods on two breast cancer DCE-MRI datasets.
BibTeX
@inproceedings{he2024tsesnet,
  author = {Jiezhou He and Xue Zhao and Zhiming Luo and Songzhi Su and Shaozi Li and Guojun Zhang},
  title = {{TSESNet: Temporal-Spatial Enhanced Breast Tumor Segmentation in DCE-MRI Using Feature Perception and Separability}},
  booktitle = {International Joint Conference on Artificial Intelligence},
  year = {2024},
  url = {https://openreview.net/forum?id=GECOkxERVO},
}
Figure