Multimodal MRI Image Fusion for Early Automatic Staging of Endometrial Cancer

Ziyu Zheng, Ye Liu, Longxiang Feng, Peizhong Liu, Haisheng Song, Lin Wang, Fang HuangSensors 2025 · DOI

TL;DR

This paper introduces a novel deep learning approach using multimodal MRI fusion to automatically stage early endometrial cancer. By combining information from sagittal, coronal, and axial MRI views with a Swin Transformer model, they achieved perfect classification accuracy, matching or exceeding radiologist performance while simplifying the process.

What Problem Does This Paper Address?

Endometrial cancer (EC) is a significant health concern, particularly for women post-menopause. Accurate staging is crucial for determining the best treatment strategy and predicting patient prognosis. Traditional staging relies on imaging techniques like MRI, but manual interpretation can be time-consuming and subject to inter-observer variability. While deep learning has shown promise in medical image analysis, previous approaches often focused on single MRI views or required laborious manual feature extraction. This paper aims to bridge this gap by developing an automated, multimodal MRI fusion method for early EC staging that is both accurate and efficient.

Methods

The core of this research is a deep learning framework built upon the Swin Transformer architecture. This model is known for its efficiency in handling high-resolution images by employing a Shifted Window Multi-head Self-Attention (SW-MSA) mechanism. This allows for capturing contextual information across different image regions without the quadratic complexity of standard self-attention.

The authors' approach involves several key steps:

  • Multimodal Fusion: Instead of relying on a single MRI view, they process images from three orthogonal planes: sagittal, coronal, and axial.
  • Image Preprocessing: MRI images are cropped, enhanced, and prepared for the model.
  • Feature Extraction: The Swin Transformer, with its SW-MSA module, extracts features from each of the three MRI planes.
  • Fusion and Classification: Features from all three planes are fused, and this combined information is fed into a classifier to determine the stage (IA or IB).
  • Ablation Studies: The researchers conducted experiments to evaluate the contribution of different components, including the Max-SA (multi-axis self-attention), SE (Squeeze-and-Excitation) module for adaptive feature weighting, and the use of medical prior knowledge (MPK).

Datasets

The study utilized a retrospective dataset of 122 patients with pathologically confirmed early-stage endometrial cancer (FIGO stage IA or IB). These patients were treated at two hospitals in Quanzhou, China, between January 2015 and December 2021. The dataset included T2-weighted MRI sequences in all three planes (sagittal, coronal, and axial). The data was split into training (97 patients) and testing (25 patients) sets.

Key Insights and Ideas

  • The Power of Multimodal Fusion: The study strongly suggests that combining information from multiple MRI views (sagittal, coronal, axial) significantly enhances classification accuracy compared to using any single view. This highlights the complementary nature of information captured in different imaging planes.
  • Swin Transformer's Efficacy: The Swin Transformer, with its efficient attention mechanism, proves highly effective for this complex medical imaging task, outperforming other deep learning models and traditional methods.
  • Synergy of Modules: The ablation studies revealed that combining the Swin Transformer with the SE module and medical prior knowledge (MPK) leads to near-perfect performance. This indicates that adaptive feature weighting and domain-specific knowledge are crucial for robust staging.
  • Automated Feature Extraction: The deep learning approach eliminates the need for manual segmentation or feature engineering, which is a significant advantage over traditional methods, reducing both time and expertise required.

Key Results

The multimodal fusion approach using the Swin Transformer achieved remarkable results:

  • Single-Plane Accuracy:
    • Sagittal: 0.988
    • Coronal: 0.96
    • Transverse: 0.94
  • Multimodal Fusion Accuracy: The combined approach achieved a perfect accuracy of 1.0 for both stage IA and IB classification.
  • Overall Performance: The final automatic classification method achieved 100% accuracy, recall, and specificity for early EC staging.
  • Comparison with Radiologists: The model's performance was comparable to, and in some aspects simpler and more precise than, that of experienced radiologists. In a test set of 60 patients, the model misclassified only two IA cases as IB, while associate director-level radiologists misclassified more cases.

Limitations and Future Work

The authors acknowledge several limitations and suggest future directions:

  • Sample Size: The study's dataset of 122 patients is relatively small. They aim to build a larger, more extensive database by collecting data from other hospitals.
  • MRI Sequences: While T2WI sequences are standard, they plan to incorporate other sequences like DWI and ADC in future studies to provide a more comprehensive diagnostic picture.
  • Radiologist Experience: While the model performed well, they note that experienced radiologists might have a slight edge in interpreting subtle cases due to their extensive experience and "global view." The proposed model is intended as an aid rather than a replacement for physicians.
  • Generalizability: Further validation on diverse patient populations and imaging equipment would be beneficial.

Summary updated

Abstract

This magnetic resonance imaging multimodal fusion study aims to automate the staging of endometrial cancer using deep learning and to compare the diagnostic performance of deep learning with that of radiologists in the staging of endometrial cancer. This study retrospectively investigated 122 patients with pathologically confirmed early EC from January 1, 2025 to December 31, 2021. Of these patients, 68 were in the International Federation of Gynecology and Obstetrics (FIGO) stage IA, and 54 were in FIGO stage IB. Based on the Swin transformer model and its proprietary SW-MSA (shift window multiple self-coherence) module, magnetic resonance imaging (MRI) images in each of the three planes (sagittal, coronal, and transverse) are cropped, enhanced, and classified, and fusion experiments in the three planes are performed simultaneously. Selecting one plane for the experiment, the accuracy of IA and IB classification was 0.988 in the sagittal, 0.96 in the coronal, and 0.94 in the transverse position, and classification accuracy after the fusion of three planes reached 1. Finally, the automatic classification method based on the Swin transformer has an accuracy of 1, a recall of 1, and a specificity of 1 for early EC classification. In this study, the multimodal fusion approach accurately classified early EC. It was comparable to what a radiologist would perform and simpler and more precise than previous methods that required segmenting followed by staging.
BibTeX
@inproceedings{zheng2025multimodal,
  author = {Ziyu Zheng and Ye Liu and Longxiang Feng and Peizhong Liu and Haisheng Song and Lin Wang and Fang Huang},
  title = {{Multimodal MRI Image Fusion for Early Automatic Staging of Endometrial Cancer}},
  booktitle = {Sensors},
  year = {2025},
  doi = {10.3390/s25092932},
}
Figure