Multi-view Evidential Learning-based Medical Image Segmentation

Chao Huang, Yushu Shi, Waikeung Wong, Chengliang Liu, Wei Wang, Zhihua Wang, Jie Wen

TL;DR

This paper introduces a novel framework for medical image segmentation that combines the strengths of traditional deep learning models and powerful vision foundation models. By leveraging multi-view learning and a new state-space model (MV-SSM), the approach extracts both domain-specific and generalizable knowledge, leading to improved segmentation accuracy. Crucially, it also incorporates evidential learning to quantify and reduce uncertainty in boundary segmentation, a critical aspect for reliable medical diagnoses.

What Problem Does This Paper Address?

Accurate medical image segmentation is vital for diagnosis, analysis, and treatment planning. While deep learning models have made significant strides, they often struggle with a "generalization bottleneck." This means models trained on specific medical datasets might not perform well on unseen data or variations. Traditional models excel at learning domain-specific features but have a limited knowledge scope. Conversely, vision foundation models (like SAM) are trained on vast datasets and capture generalizable knowledge, but they lack the fine-tuned understanding of specific medical domains. Directly applying foundation models to medical images often leads to suboptimal performance due to domain differences, and fine-tuning them is prohibitively expensive due to the scarcity of high-quality labeled medical data. Furthermore, accurately segmenting boundaries in medical images is inherently challenging due to ambiguity, leading to segmentation uncertainty that can impact diagnostic reliability.

Methods

The proposed framework tackles these challenges through a multi-pronged approach:

  • Multi-view State Space Model (MV-SSM): This novel module is designed to extract task-related knowledge by modeling cross-view contextual dependencies. It utilizes Mamba, a state-space model known for its efficiency in handling long sequences, to bridge the gap between domain-specific and generalizable features.
  • Mamba Residual Channel Cross-Fusion Transformer (Mamba Residual CCT): This component is introduced to enrich domain-specific knowledge. It combines the strengths of Channel-wise Cross Fusion Transformers (CCT) with Mamba, incorporating a residual connection to preserve original information and a window-based channel attention mechanism to capture both global and local features effectively.
  • Evidential Learning: To address segmentation uncertainty, particularly at boundaries, the paper employs evidential learning. Instead of a standard softmax output, the model uses a variational Dirichlet distribution to characterize the probability distribution of segmentation results. This allows for the quantification of uncertainty based on collected evidence. By optimizing the parameters of this Dirichlet distribution, the model actively reduces boundary segmentation uncertainty.

The overall architecture involves using a vision foundation model (SAM) as a generalizable feature extractor and a separate domain-specific encoder. The features from both are then fed into the MV-SSM to extract task-related knowledge. A domain-specific decoder collects evidence, which is then used by the evidential learning module to quantify and minimize uncertainty.

Datasets

The proposed method is evaluated on four publicly available medical image segmentation datasets:

  • MoNuSeg: Contains 30 digital microscopic tissue images.
  • GlaS: Includes 85 images for training and 80 for testing.
  • TNBC: Features 50 images with 4,022 annotated cells.
  • DRIVE: A retinal vessel segmentation dataset with 40 images (20 for training, 20 for testing).

Key Insights and Ideas

  • Synergy of Foundation and Domain-Specific Models: The core insight is that combining the broad, generalizable knowledge from vision foundation models with the focused, domain-specific knowledge from traditional deep learning models is crucial for robust medical image segmentation.
  • Mamba for Cross-View Dependencies: The adoption of Mamba within the MV-SSM is a key innovation, enabling efficient modeling of complex relationships between features extracted from different views or modalities.
  • Evidential Learning for Uncertainty Quantification: Moving beyond simple probability outputs, using evidential learning with a Dirichlet distribution provides a principled way to quantify and actively reduce segmentation uncertainty, which is critical for clinical trust.
  • Balancing Global and Local Information: The Mamba Residual CCT's design highlights the importance of simultaneously capturing both fine-grained local details and broader contextual information for accurate segmentation.

Key Results

The proposed method consistently outperforms existing state-of-the-art methods across all evaluated datasets. For instance, on the MoNuSeg dataset, it achieved an 81.88% Dice score and 69.45% mIoU, surpassing previous benchmarks. Similar improvements were observed on the GlaS (90.59% Dice, 83.70% mIoU) and TNBC (83.75% Dice, 72.23% mIoU) datasets. Qualitative results also demonstrate superior boundary segmentation. Ablation studies confirm the effectiveness of each proposed component, particularly the evidential learning module, which significantly boosts performance by minimizing boundary uncertainty.

Limitations and Future Work

The paper does not explicitly detail limitations or future work beyond the general statement that the potential of Mamba in multi-view learning is an area that has not been thoroughly explored. However, based on the methodology, potential future directions could include:

  • Exploring other vision foundation models or different Mamba variants.
  • Investigating more advanced evidential learning techniques for uncertainty quantification.
  • Applying the framework to a wider range of medical imaging modalities and segmentation tasks.
  • Further research into optimizing the trade-off between domain-specific and generalizable knowledge extraction.

Summary updated

BibTeX
@inproceedings{huang2025multi,
  author = {Chao Huang and Yushu Shi and Waikeung Wong and Chengliang Liu and Wei Wang and Zhihua Wang and Jie Wen},
  title = {{Multi-view Evidential Learning-based Medical Image Segmentation}},
  booktitle = {AAAI Conference on Artificial Intelligence},
  year = {2025},
  doi = {10.1609/aaai.v39i16.33911},
}
Figure