NaMa: Neighbor-Aware Multi-Modal Adaptive Learning for Prostate Tumor Segmentation on Anisotropic MR Images

Runqi Meng, Xiao Zhang, Shijie Huang, Yuning Gu, Guiqin Liu, Guangyu Wu, Nizhuan Wang, Kaicong Sun, Dinggang ShenAAAI 2024 · DOI

Abstract

Accurate prostate tumor segmentation from multi-modal MRI is critical for clinical diagnosis but faces challenges due to subject-specific modality reliability and the anisotropic nature of clinical "thick-slice" scans. We present NaMa (Neighbor-aware Multi-modal adaptive learning), a novel framework designed to adaptively weigh MRI modalities and enforce inter-slice consistency. NaMa incorporates a Modality-Informativeness Adaptive Learning (MIAL) module to handle varying modality quality and an Activation Mapping Guidance (AMG) module to leverage neighboring slice information for refined segmentation. Evaluated on in-house and public (PICAI) datasets, NaMa significantly outperforms state-of-the-art methods, achieving a Dice Similarity Coefficient (DSC) of 63.76% on the in-house dataset.

Introduction

Prostate cancer remains a leading cause of mortality among men. Magnetic Resonance Imaging (MRI), specifically T2-weighted (T2WI), Diffusion-weighted (DWI), and Apparent Diffusion Coefficient (ADC) maps, is the gold standard for non-invasive assessment. However, automated segmentation is hindered by two primary factors. First, the diagnostic value of each modality varies per patient; noise or artifacts in one modality can degrade standard fusion models. Second, clinical protocols often produce anisotropic images with high in-plane resolution but low through-plane resolution, causing 2D models to lose spatial context and 3D models to struggle with data "stretching."

Methods

The NaMa network addresses these challenges through a two-stage adaptive learning process.

Modality-Informativeness Adaptive Learning (MIAL)

Rather than simple concatenation, the MIAL module employs a gating mechanism to dynamically assess the informativeness of T2WI, DWI, and ADC features for each specific subject. By calculating cross-modality correlations, the module suppresses noisy or uninformative channels, ensuring the network prioritizes the most reliable data. A random mask strategy is further used during training to enhance feature robustness.

Activation Mapping Guidance (AMG)

To overcome the limitations of anisotropic "thick-slice" data, the AMG module enforces 3D spatial consistency. It monitors the network's focus by comparing activation maps of adjacent slices using the Wasserstein distance. High distances indicate low confidence in a particular slice's segmentation. In such cases, the module uses an attention mechanism to integrate features from more confident neighboring slices, smoothing the final 3D volume.

Architecture of the NaMa network, featuring the MIAL and AMG modules.

Results and Discussion

NaMa was evaluated on an in-house (IH) dataset of 355 patients and the public PICAI dataset of 215 patients.

Quantitative Performance

NaMa achieved a DSC of 63.76% on the IH dataset, outperforming the previous state-of-the-art CSAD (59.06%). Preliminary studies indicated that modality importance is highly subjective, with only ~5% of cases performing best using all modalities simultaneously. NaMa's adaptive fusion addresses this by selecting the optimal data per patient.

Qualitative Analysis

Visual inspections confirm that NaMa produces more anatomically consistent tumor volumes compared to traditional 2D or 3D fusion methods. The model effectively handles cases where specific modalities are degraded, maintaining high segmentation quality where other models fail.

Quantitative comparison of model inputs and qualitative segmentation results on PICAI and IH datasets.

Conclusion

The NaMa framework successfully addresses modality variability and anisotropy in prostate MRI segmentation. By adaptively selecting informative features and leveraging neighbor-aware guidance, it provides a robust solution for clinical applications. Future work will focus on integrating NaMa into clinical workflows for surgical and radiotherapy planning and exploring its predictive value for long-term patient prognosis.

Summary updated

BibTeX
@inproceedings{meng2024nama,
  author = {Runqi Meng and Xiao Zhang and Shijie Huang and Yuning Gu and Guiqin Liu and Guangyu Wu and Nizhuan Wang and Kaicong Sun and Dinggang Shen},
  title = {{NaMa: Neighbor-Aware Multi-Modal Adaptive Learning for Prostate Tumor Segmentation on Anisotropic MR Images}},
  booktitle = {AAAI Conference on Artificial Intelligence},
  year = {2024},
  doi = {10.1609/aaai.v38i5.28215},
}
Figure