UPMAD-Net: A Brain Tumor Segmentation Network with Uncertainty Guidance and Adaptive Multimodal Feature Fusion

Ziyu Jia, Yao Ni, Danyang Sun, Chuang Han, Yanting Li, Jiaofen Nan, Fubao Zhu, Chen Zhao, Weihua ZhouarXiv 2025 · DOI · arXiv

Abstract

Accurate brain tumor segmentation is critical for diagnosis and treatment planning but remains challenging due to irregular tumor shapes and vague boundaries. We propose UPMAD-Net, a 3D segmentation network that integrates deep learning with prior knowledge from region-growing algorithms. The architecture features a multi-scale feature fusion (MSFF) module and adaptive attention mechanisms (AAM) to capture global context and multi-scale details. To improve robustness in ambiguous regions, Monte Carlo Dropout is used for uncertainty estimation. Evaluated on BraTS2021, UPMAD-Net achieved Dice scores of 89.18% (ET), 93.67% (WT), and 91.23% (TC), outperforming several state-of-the-art methods with a significantly smaller parameter footprint (0.34M).

Introduction

Brain tumor segmentation is a cornerstone of neuro-oncology, yet it is notoriously difficult. Tumors are heterogeneous, often exhibiting irregular shapes and "fuzzy" boundaries that blend into healthy tissue. While standard Convolutional Neural Networks (CNNs) have revolutionized the field, they often struggle with long-range dependencies due to local receptive fields. Conversely, Transformer-based models capture global context but are computationally expensive and data-hungry. Furthermore, most models are "overconfident," providing deterministic masks without indicating where segmentation might be unreliable—a critical flaw for clinical decision-making.

UPMAD-Net (Uncertainty-guided Prior-integrated Multiscale Attention-based Decoder Network) addresses these challenges by combining traditional image processing "priors" with an adaptive, lightweight multi-scale architecture.

Methodology

The UPMAD-Net framework utilizes a four-stage encoder-decoder structure inspired by U-Net, enhanced with several novel components to improve accuracy and efficiency.

The UPMAD-Net framework integrates a traditional image processing prior with a multi-scale U-Net architecture.

Prior Knowledge Module

Before the neural network processes the MRI data, the system applies Otsu thresholding and a 3D region-growing algorithm specifically to the FLAIR modality. This generates a coarse "hint" mask, which is concatenated as a fifth input channel. This structural guidance helps the model stabilize its predictions, especially when combined with more complex attention modules.

Multi-Scale Feature Fusion (MSFF)

To handle tumors of varying sizes, the encoder employs MSFF modules. These modules utilize parallel convolutional branches with different kernel sizes and dilations (1x1x1, 3x3x3, and dilated 3x3x3). This allows the network to capture both fine details and broader context without the computational overhead of massive kernels.

The MSFF module uses parallel branches with varying dilations to capture features at multiple scales.

Adaptive Attention and Uncertainty Estimation

Integrated into the decoder, the Adaptive Attention Mechanism (AAM) dynamically reweights voxel-level features, focusing the model on relevant tumor regions while suppressing background noise.

During inference, the model employs Monte Carlo (MC) Dropout. By executing 20 stochastic forward passes, the model generates an average prediction and an "uncertainty map" (variance). This map highlights ambiguous regions, such as infiltrative boundaries, providing clinicians with a "confidence score" that flags areas potentially requiring manual review.

Results and Discussion

The model was evaluated on the BraTS2021 (1,251 cases) and BraTS2019 (335 cases) datasets, covering four MRI modalities (T1, T1ce, T2, and FLAIR).

Performance Metrics

UPMAD-Net outperformed several state-of-the-art (SOTA) models, including Swin-UnetR and MPEDA-Net. On the BraTS2021 test set, it achieved Dice scores of 89.18% for the Enhancing Tumor (ET), 93.67% for the Whole Tumor (WT), and 91.23% for the Tumor Core (TC). It also demonstrated significantly lower Hausdorff Distance (HD) values, indicating tighter alignment with ground-truth boundaries.

Computational Efficiency

A key highlight of UPMAD-Net is its efficiency. The model contains only 0.34M parameters and requires 20.01G FLOPs. In comparison, many competing SOTA models use 10x to 50x more parameters (e.g., RALU-Net at 8.52M parameters), making UPMAD-Net highly suitable for resource-constrained clinical environments.

Conclusion

This study introduced UPMAD-Net, a novel 3D brain tumor segmentation network that leverages prior knowledge and uncertainty estimation to improve robustness and performance. By incorporating dilated convolutions and adaptive attention, the model achieves high accuracy with a remarkably small parameter footprint. Future work will explore more efficient Transformer integrations and optimization of 3D convolutional modules to further enhance global contextual modeling without sacrificing efficiency.

Summary updated

Abstract

Background: Brain tumor segmentation has a significant impact on the diagnosis and treatment of brain tumors. Accurate brain tumor segmentation remains challenging due to their irregular shapes, vague boundaries, and high variability. Objective: We propose a brain tumor segmentation method that combines deep learning with prior knowledge derived from a region-growing algorithm. Methods: The proposed method utilizes a multi-scale feature fusion (MSFF) module and adaptive attention mechanisms (AAM) to extract multi-scale features and capture global contextual information. To enhance the model's robustness in low-confidence regions, the Monte Carlo Dropout (MC Dropout) strategy is employed for uncertainty estimation. Results: Extensive experiments demonstrate that the proposed method achieves superior performance on Brain Tumor Segmentation (BraTS) datasets, significantly outperforming various state-of-the-art methods. On the BraTS2021 dataset, the test Dice scores are 89.18% for Enhancing Tumor (ET) segmentation, 93.67% for Whole Tumor (WT) segmentation, and 91.23% for Tumor Core (TC) segmentation. On the BraTS2019 validation set, the validation Dice scores are 87.43%, 90.92%, and 90.40% for ET, WT, and TC segmentation, respectively. Ablation studies further confirmed the contribution of each module to segmentation accuracy, indicating that each component played a vital role in overall performance improvement. Conclusion: This study proposed a novel 3D brain tumor segmentation network based on the U-Net architecture. By incorporating the prior knowledge and employing the uncertainty estimation method, the robustness and performance were improved. The code for the proposed method is available at https://github.com/chenzhao2023/UPMAD_Net_BrainSeg.
BibTeX
@misc{jia2025upmad,
  author = {Ziyu Jia and Yao Ni and Danyang Sun and Chuang Han and Yanting Li and Jiaofen Nan and Fubao Zhu and Chen Zhao and Weihua Zhou},
  title = {{UPMAD-Net: A Brain Tumor Segmentation Network with Uncertainty Guidance and Adaptive Multimodal Feature Fusion}},
  year = {2025},
  eprint = {2505.03494},
  archivePrefix = {arXiv},
  url = {https://arxiv.org/abs/2505.03494},
}
Figure