Abstract
Deep learning models for medical image segmentation are increasingly powerful but remain susceptible to "silent failures" when encountering out-of-distribution data. This paper introduces QCResUNet, a multi-task framework designed to provide a robust safety net for automated segmentation. By simultaneously predicting global quality metrics (Dice scores) and generating voxel-level segmentation error maps (SEMs), QCResUNet identifies not only if a segmentation has failed but exactly where the errors are located. Evaluated across brain tumor and cardiac MRI datasets, the model demonstrates high correlation with ground truth quality metrics and a 127.5% improvement in error localization over traditional uncertainty-based methods.
Introduction
Automated medical image segmentation is a cornerstone of modern radiology, particularly for complex tasks like brain tumor delineation. However, the clinical adoption of these models is hindered by their lack of transparency regarding failure modes. When a model encounters data that differs from its training set—due to different scanner protocols or patient demographics—it may produce plausible-looking but inaccurate results.
Existing Automated Quality Control (QC) tools typically provide a single subject-level score. While useful for flagging a scan, these scores lack the spatial context necessary for a clinician to efficiently correct the output. QCResUNet addresses this by providing a dual-output system: a global quality assessment and a localized error map.
Methodology
The QCResUNet Architecture
QCResUNet utilizes a 3D U-shaped architecture optimized for joint regression and segmentation. The framework consists of:
- ResNet-34 Encoder: A modified encoder extracts high-level features from the input imaging modalities and the candidate segmentation mask.
- Multi-Task Heads: The model branches into two paths. One performs regression to predict global metrics like the Dice Similarity Coefficient (DSC) and Normalized Surface Dice (NSD). The other uses a decoder to map features back to the original image dimensions, producing voxel-level Segmentation Error Maps (SEMs).
- Efficient Channel Attention (ECA): To better correlate imaging features with the segmentation mask, the authors integrated ECA. This mechanism allows the model to focus on specific tissue-modality relationships, such as identifying errors in the peritumoral edema versus the necrotic core.
SegGen: Generating Training Data
A significant challenge in training QC models is the lack of "failed" segmentations with known ground truth errors. The authors developed SegGen, a four-step data generation strategy. SegGen creates a diverse library of realistic errors by using data augmentation and segmentations produced by models at various stages of under-training. This ensures the QC model learns to recognize a wide spectrum of segmentation failures.
Loss Functions
The model is trained using a weighted multi-task loss function. Global quality prediction is optimized via Mean Absolute Error (MAE), while the voxel-level error maps are trained using a combination of Dice and Cross-Entropy losses to ensure spatial accuracy.
Experimental Results
Datasets and Generalization
The model was validated on internal data (BraTS 2021, n=1,251) and tested for generalization on external, out-of-distribution datasets, including the BraTS-SSA (Sub-Saharan Africa) and WUSM (Washington University School of Medicine) cohorts. To demonstrate versatility, the model was also applied to the ACDC cardiac MRI dataset.
Subject-Level Performance
On the BraTS test set, QCResUNet achieved a Pearson correlation ($r$) of 0.968 for DSC prediction and 0.958 for NSD. This indicates a near-perfect linear relationship between the model's predicted quality and the actual segmentation accuracy, significantly outperforming standard ResNet baselines.
Voxel-Level Localization
The model's ability to pinpoint errors (DSC$_{SEM}$) reached 0.769 for brain tumors. This represents a 127.5% improvement over traditional uncertainty-based methods. Unlike uncertainty maps, which are often noisy and highlight boundaries regardless of accuracy, QCResUNet's SEMs specifically highlight areas of misclassification.
Computational Efficiency
Despite its dual-task capability, QCResUNet is highly efficient. With a FLOP count of 296.64G, it is comparable to a standard ResNet-50 and significantly more lightweight than a standard 3D UNet (670.24G), making it suitable for integration into clinical pipelines.
Discussion and Limitations
While QCResUNet sets a new benchmark for segmentation QC, it faces a "high-quality ceiling." When the initial segmentation is nearly perfect, localizing the few remaining single-voxel errors becomes significantly more difficult than identifying large-scale failures. Additionally, the model currently relies on the presence of all four standard MRI modalities for brain tumors; performance degrades if modalities like FLAIR or T1c are missing.
Conclusion
QCResUNet provides a comprehensive solution for medical image quality control by combining global performance metrics with local error visualization. By moving beyond simple uncertainty and providing actionable spatial feedback, this framework offers a viable path toward safer and more reliable AI integration in clinical radiology.
Summary updated