Introduction
Deep Learning (DL) models have shown remarkable success in automating medical image analysis, particularly in segmentation tasks like tumor or polyp detection. However, their clinical deployment is hindered by sensitivity to "out-of-distribution" (OOD) data—samples that deviate from training data due to hardware variations, noise, or rare pathologies. Traditionally, Quality Control (QC) is split into two silos: Input QC (detecting corrupted images) and Output QC (assessing prediction reliability). This paper proposes a unified framework that bridges these domains, providing a four-quadrant stratification system to flag high-risk cases for human intervention.
A Unified Taxonomy and Framework
The authors began by analyzing 95 existing QC studies, categorizing them by domain and methodology. They identified a critical gap: while input and output quality are intertwined, they are rarely managed jointly.
To address this, the paper introduces an ensemble-based framework that requires no specialized QC training. The system utilizes an ensemble of $K$ segmentation models to perform dual-faceted checks:
- Input QC via Latent Space Analysis: The framework employs Globally Normalized Mahalanobis Distance (GNMD). By monitoring the internal activations of the neural network, the system can determine if a test image's feature representation is "far" from the training distribution.
- Output QC via Ensemble Agreement: The Ensemble Prediction Agreement (EPA) measures the consistency across the $K$ models. High Dice overlap between masks suggests reliability, while significant disagreement flags potential failures.
Stratification of the Prediction Space
The core innovation is the division of predictions into four distinct regions based on QC passes/fails:
- Region A (Optimum): Both input and output pass; high confidence.
- Region B (Robust): Input fails (e.g., noisy image), but output passes; the model demonstrates resilience.
- Region C (Dysfunctional): Input passes, but output fails; these "silent failures" are the most dangerous in clinical settings.
- Region D (Divergent): Both fail; highest priority for manual review.
Experimental Validation
The framework was tested on two major tasks: 2D endoscopic polyp segmentation and 3D brain tumor segmentation (using BraTS 2023). To simulate real-world challenges, the researchers used datasets from diverse geographical locations and age groups, alongside synthetic degradations like motion blur and noise.
The results validated the stratification strategy. Region A consistently yielded the highest performance (e.g., 0.88 Dice for polyps), while Region D successfully isolated the most severe failures (Dice scores as low as 0.32). Interestingly, the "Robust" Region B demonstrated that models can often handle moderate noise, suggesting that OOD detection alone is too conservative for clinical workflows.
Conclusion
By unifying input and output quality control, this framework provides a practical "triage" system for medical AI. It highlights that while input quality is important, the ultimate goal is reliable output. Future work will focus on optimizing the computational overhead of ensembles and refining thresholding strategies for specific clinical use cases.
Summary updated