Demystifying Deep Learning-based Brain Tumor Segmentation with 3D UNets and Explainable AI (XAI): A Comparative Analysis

Ming Jie Ong, Sze Yinn Ung, Sim Kuan Goh, Jimmy Y. Zhong

Abstract

The current study investigated the use of Explainable Artificial Intelligence (XAI) to improve the accuracy of brain tumor segmentation in MRI images, with the goal of assisting physicians in clinical decision-making. We compared the performance of three deep learning models—UNet, Residual UNet (ResUNet), and Attention UNet (AttUNet)—on the BraTS2020 dataset. XAI techniques, including Grad-CAM and attention-based visualization, were employed to clarify model decisions. Notably, ResUNet outperformed the other models in Dice and Jaccard similarity scores, accuracy, and inference speed. These results demonstrate ResUNet as the best-performing model for automated brain tumor segmentation in clinical assessments.

Introduction

Automated brain tumor segmentation is a high-stakes task in medical imaging. While deep learning models can identify tumor boundaries in MRI scans, they often act as "black boxes," making it difficult for physicians to trust their outputs. Furthermore, many existing models rely on 2D slices, which lose the vital volumetric context of the human brain. This research addresses two main gaps: the Dimensionality Gap, by moving from 2D to 3D convolutions to capture spatial relationships, and the Trust Gap, by using Explainable AI (XAI) to visualize model focus.

Methodology

The study evaluates three distinct 3D architectures based on the UNet framework:

  • 3D UNet: The baseline symmetrical encoder-decoder model.
  • 3D ResUNet: This model incorporates residual blocks to facilitate gradient flow and prevent the "vanishing gradient" problem, allowing the network to learn finer structural details.
  • 3D AttUNet: This architecture adds Attention Gates and the Convolutional Block Attention Module (CBAM) to the skip connections, theoretically helping the model focus only on relevant tumor regions.
The 3D ResUNet architecture incorporates residual blocks to improve gradient flow and feature extraction.

Models were trained using a hybrid BCE-Dice loss, combining Binary Cross Entropy for pixel-wise probability and Dice loss for overlap accuracy. To address the black-box nature of these models, Grad-CAM was used to produce heatmaps of the final convolutional layers, while attention-based visualization inspected the internal weights of the AttUNet’s attention modules.

The 3D AttUNet architecture adds complexity through Attention Gates and 3D CBAM modules.

Experimental Setup

The researchers utilized the BraTS2020 (Brain Tumor Segmentation) public dataset, consisting of 368 3D MRI brain volumes. The labels include three distinct regions: the necrotic/non-enhancing tumor core, peritumoral edema, and enhancing tumor. Images were preprocessed by cropping to 170x170x100 to remove empty space and optimize computational efficiency.

Results and Discussion

Performance and Efficiency

Surprisingly, the most complex model (AttUNet) was not the most effective. While it converged in fewer epochs, it required massive computational resources—taking 13 times longer to train per epoch than the other models.

ResUNet emerged as the top performer across all metrics:

  • Validation Dice Score: 0.897 (compared to 0.887 for AttUNet and 0.881 for UNet).
  • Validation Jaccard Score: 0.818.
  • Test Accuracy: 99.7%.
  • Inference Speed: ResUNet was the fastest, processing a volume in just 8.45 seconds, significantly outperforming the standard UNet's 25.68 seconds.

Explainable AI Insights

Grad-CAM visualizations provided a critical "Aha" moment. Heatmaps revealed that ResUNet focused sharply on specific tumor subregions. In contrast, AttUNet’s focus appeared broader and more "smeared" over the tumor area. This suggests that the attention mechanism may have been too broad, potentially leading to slight overfitting or a loss of fine-grained boundary detail. Residual connections proved more effective for medical imaging by allowing the network to perform "identity mapping," skipping layers that do not add value and speeding up both training and inference.

Conclusion

This study identifies ResUNet as the optimal balance of accuracy and efficiency for 3D brain tumor segmentation. While attention mechanisms offer theoretical benefits, they did not translate to superior clinical performance in this context and came at a high computational cost. Future work will explore larger datasets, Squeeze-and-Excitation (SE) blocks for channel-wise attention, and uncertainty estimation to provide clinicians with a confidence metric alongside segmentation masks.

Summary updated

Abstract

The current study investigated the use of Explainable Artificial Intelligence (XAI) to improve the accuracy of brain tumor segmentation in MRI images, with the goal of assisting physicians in clinical decision-making. The study focused on applying UNet models for brain tumor segmentation and using the XAI techniques of Gradient-weighted Class Activation Mapping (Grad-CAM) and attention-based visualization to enhance the understanding of these models. Three deep learning models - UNet, Residual UNet (ResUNet), and Attention UNet (AttUNet) - were evaluated to identify the best-performing model. XAI was employed with the aims of clarifying model decisions and increasing physicians' trust in these models. We compared the performance of two UNet variants (ResUNet and AttUNet) with the conventional UNet in segmenting brain tumors from the BraTS2020 public dataset and analyzed model predictions with Grad-CAM and attention-based visualization. Using the latest computer hardware, we trained and validated each model using the Adam optimizer and assessed their performance with respect to: (i) training, validation, and inference times, (ii) segmentation similarity coefficients and loss functions, and (iii) classification performance. Notably, during the final testing phase, ResUNet outperformed the other models with respect to Dice and Jaccard similarity scores, as well as accuracy, recall, and F1 scores. Grad-CAM provided visuospatial insights into the tumor subregions each UNet model focused on while attention-based visualization provided valuable insights into the working mechanisms of AttUNet's attention modules. These results demonstrated ResUNet as the best-performing model and we conclude by recommending its use for automated brain tumor segmentation in future clinical assessments. Our source code and checkpoint are available at https://github.com/ethanong98/MultiModel-XAI-Brats2020
BibTeX
@misc{ong2025demystifying,
  author = {Ming Jie Ong and Sze Yinn Ung and Sim Kuan Goh and Jimmy Y. Zhong},
  title = {{Demystifying Deep Learning-based Brain Tumor Segmentation with 3D UNets and Explainable AI (XAI): A Comparative Analysis}},
  year = {2025},
  eprint = {2510.07785},
  archivePrefix = {arXiv},
  url = {https://arxiv.org/abs/2510.07785},
}
Figure