QCResUNet: Joint Subject-Level and Voxel-Level Prediction of Segmentation Quality

Peijie Qiu, Satrajit Chakrabarty, Phuc Nguyen, Soumyendu Sekhar Ghosh, Aristeidis Sotiras

Abstract

Deep learning models for medical image segmentation are increasingly powerful but remain susceptible to "silent failures" when encountering out-of-distribution data. This paper introduces QCResUNet, a multi-task framework designed to provide a robust safety net for automated segmentation. By simultaneously predicting global quality metrics (Dice scores) and generating voxel-level segmentation error maps (SEMs), QCResUNet identifies not only if a segmentation has failed but exactly where the errors are located. Evaluated across brain tumor and cardiac MRI datasets, the model demonstrates high correlation with ground truth quality metrics and a 127.5% improvement in error localization over traditional uncertainty-based methods.

Introduction

Automated medical image segmentation is a cornerstone of modern radiology, particularly for complex tasks like brain tumor delineation. However, the clinical adoption of these models is hindered by their lack of transparency regarding failure modes. When a model encounters data that differs from its training set—due to different scanner protocols or patient demographics—it may produce plausible-looking but inaccurate results.

Existing Automated Quality Control (QC) tools typically provide a single subject-level score. While useful for flagging a scan, these scores lack the spatial context necessary for a clinician to efficiently correct the output. QCResUNet addresses this by providing a dual-output system: a global quality assessment and a localized error map.

Methodology

The QCResUNet Architecture

QCResUNet utilizes a 3D U-shaped architecture optimized for joint regression and segmentation. The framework consists of:

  • ResNet-34 Encoder: A modified encoder extracts high-level features from the input imaging modalities and the candidate segmentation mask.
  • Multi-Task Heads: The model branches into two paths. One performs regression to predict global metrics like the Dice Similarity Coefficient (DSC) and Normalized Surface Dice (NSD). The other uses a decoder to map features back to the original image dimensions, producing voxel-level Segmentation Error Maps (SEMs).
  • Efficient Channel Attention (ECA): To better correlate imaging features with the segmentation mask, the authors integrated ECA. This mechanism allows the model to focus on specific tissue-modality relationships, such as identifying errors in the peritumoral edema versus the necrotic core.

SegGen: Generating Training Data

A significant challenge in training QC models is the lack of "failed" segmentations with known ground truth errors. The authors developed SegGen, a four-step data generation strategy. SegGen creates a diverse library of realistic errors by using data augmentation and segmentations produced by models at various stages of under-training. This ensures the QC model learns to recognize a wide spectrum of segmentation failures.

Loss Functions

The model is trained using a weighted multi-task loss function. Global quality prediction is optimized via Mean Absolute Error (MAE), while the voxel-level error maps are trained using a combination of Dice and Cross-Entropy losses to ensure spatial accuracy.

Experimental Results

Datasets and Generalization

The model was validated on internal data (BraTS 2021, n=1,251) and tested for generalization on external, out-of-distribution datasets, including the BraTS-SSA (Sub-Saharan Africa) and WUSM (Washington University School of Medicine) cohorts. To demonstrate versatility, the model was also applied to the ACDC cardiac MRI dataset.

Subject-Level Performance

On the BraTS test set, QCResUNet achieved a Pearson correlation ($r$) of 0.968 for DSC prediction and 0.958 for NSD. This indicates a near-perfect linear relationship between the model's predicted quality and the actual segmentation accuracy, significantly outperforming standard ResNet baselines.

Voxel-Level Localization

The model's ability to pinpoint errors (DSC$_{SEM}$) reached 0.769 for brain tumors. This represents a 127.5% improvement over traditional uncertainty-based methods. Unlike uncertainty maps, which are often noisy and highlight boundaries regardless of accuracy, QCResUNet's SEMs specifically highlight areas of misclassification.

Computational Efficiency

Despite its dual-task capability, QCResUNet is highly efficient. With a FLOP count of 296.64G, it is comparable to a standard ResNet-50 and significantly more lightweight than a standard 3D UNet (670.24G), making it suitable for integration into clinical pipelines.

Discussion and Limitations

While QCResUNet sets a new benchmark for segmentation QC, it faces a "high-quality ceiling." When the initial segmentation is nearly perfect, localizing the few remaining single-voxel errors becomes significantly more difficult than identifying large-scale failures. Additionally, the model currently relies on the presence of all four standard MRI modalities for brain tumors; performance degrades if modalities like FLAIR or T1c are missing.

Conclusion

QCResUNet provides a comprehensive solution for medical image quality control by combining global performance metrics with local error visualization. By moving beyond simple uncertainty and providing actionable spatial feedback, this framework offers a viable path toward safer and more reliable AI integration in clinical radiology.

Summary updated

Abstract

Deep learning has achieved state-of-the-art performance in automated brain tumor segmentation from magnetic resonance imaging (MRI) scans. However, the unexpected occurrence of poor-quality outliers, especially in out-of-distribution samples, hinders their translation into patient-centered clinical practice. Therefore, it is important to develop automated tools for large-scale segmentation quality control (QC). However, most existing QC methods targeted cardiac MRI segmentation which involves a single modality and a single tissue type. Importantly, these methods only provide a subject-level segmentation-quality prediction, which cannot inform clinicians where the segmentation needs to be refined. To address this gap, we proposed a novel network architecture called QCResUNet that simultaneously produces segmentation-quality measures as well as voxel-level segmentation error maps for brain tumor segmentation QC. To train the proposed model, we created a wide variety of segmentation-quality results by using i) models that have been trained for a varying number of epochs with different modalities; and ii) a newly devised segmentation-generation method called SegGen. The proposed method was validated on a large public brain tumor dataset with segmentations generated by different methods, achieving high performance on the prediction of segmentation-quality metric as well as voxel-wise localization of segmentation errors. The implementation will be publicly available at https://github.com/peijie-chiu/QC-ResUNet .
BibTeX
@inproceedings{qiu2023qcresunet,
  author = {Peijie Qiu and Satrajit Chakrabarty and Phuc Nguyen and Soumyendu Sekhar Ghosh and Aristeidis Sotiras},
  title = {{QCResUNet: Joint Subject-Level and Voxel-Level Prediction of Segmentation Quality}},
  booktitle = {International Conference on Medical Image Computing and Computer-Assisted Intervention},
  year = {2023},
  doi = {10.1007/978-3-031-43901-8\_17},
}

Loading related content...

Figure