When evaluating a medical image segmentation model, computer vision researchers typically rely on geometric metrics: the Dice Similarity Coefficient (DSC), Intersection over Union (IoU), or Hausdorff Distance. These metrics treat every pixel or voxel with equal weight. If a model achieves a 95% Dice score, it is celebrated as a success.
However, in a clinical setting, a 5% error is not just a statistical deviation—it can be the difference between a successful treatment and a fatal complication. To understand why segmentation works or fails in practice, we must look past pixel-level overlaps and examine how geometric errors propagate into downstream clinical decisions.
The Gap Between Geometry and Dosimetry in Radiotherapy
In radiation therapy, a linear accelerator delivers targeted radiation to a tumor while attempting to spare surrounding Organs at Risk (OARs). This optimization process relies entirely on the spatial boundaries defined by segmentation. If a segmentation model under-segments a tumor boundary by just a few millimeters, the resulting radiation plan will under-dose the malignancy, risking cancer recurrence. Conversely, over-segmenting into an adjacent critical structure—like an optic nerve or a healthy bowel—can lead to irreversible tissue damage.
Evaluating models purely on geometric overlap hides these critical consequences. In Comparison of Vendor-Pretrained and Custom-Trained Deep Learning Segmentation Models for Head-and-Neck, Breast, and Prostate Cancers, Xinru Chen et al. demonstrated this gap by introducing dosimetric evaluations. By measuring mean and maximum dose errors ($\Delta D_{mean}$ and $\Delta D_{max}$) against clinically approved contours, the authors showed that minor geometric discrepancies between models translate directly into measurable, potentially dangerous variations in the radiation dose delivered to critical structures.
This risk is magnified when models encounter anatomical anomalies. In Rethinking Abdominal Organ Segmentation (RAOS) in the clinical scenario: A robustness evaluation benchmark with challenging cases, Xiangde Luo et al. uncovered a phenomenon known as "organ hallucination." When a deep learning model segments a CT scan of a patient who has had an organ (such as a gallbladder or kidney) surgically removed, it often "hallucinates" the missing organ, filling the empty space with a false prediction. In radiotherapy planning, if a model incorrectly identifies a resected area as a healthy, vulnerable organ, the optimization algorithm will steer the radiation beams away from that zone, potentially under-dosing the surrounding target area where recurrent tumor cells might reside.
Downstream Biomarkers and Volumetric Downfalls
Beyond radiation planning, segmentation serves as the foundational step for calculating downstream clinical biomarkers. Clinicians rarely care about the segmentation mask itself; instead, they care about the metrics derived from it, such as organ volume, tumor burden, or tissue density.
Consider the calculation of Bone Mineral Density (BMD) from Dual-Energy X-ray Absorptiometry (DXA) images. Dildar Hussain demonstrates that the clinical utility of BMD calculations is entirely dependent on the precision of femur boundary segmentation. If image noise or low contrast causes the model to misjudge the bone boundary by even a small margin, the resulting density calculation becomes skewed, directly undermining the diagnosis of osteoporosis and the assessment of fracture risks.
Similarly, anatomical segmentation in diseased states directly impacts disease monitoring. In CF-Seg: Counterfactuals meet Segmentation, Raghav Mehta highlights that when segmentation models fail to accurately distinguish between healthy tissue and pathological patterns, the resulting volume measurements become skewed. Because these volumetric metrics dictate clinical staging and treatment adjustments, geometric inaccuracy propagates directly into incorrect medical assessments.
Why Standard Metrics Fail to Capture Clinical Reality
The core reason standard metrics fail is that they lack asymmetric risk awareness. In a standard Dice loss function, missing a pixel on the outer edge of a tumor carries the exact same penalty as missing a pixel on the inner edge bordering a critical blood vessel.
In reality, medical errors are highly asymmetric:
- False Positives (Over-segmentation): Can lead to unnecessary healthy tissue avoidance in surgery or radiation, or cause false alarms that trigger invasive secondary biopsies.
- False Negatives (Under-segmentation): Can leave residual tumor tissue behind during a resection or cause a clinician to miss an early-stage malignancy entirely.
To bridge this gap, the field is shifting toward evaluating models within their specific clinical context. Whether by measuring dosimetric impact in radiotherapy or tracking biomarker variance in diagnostics, the true measure of a segmentation model's accuracy is not its performance on a pixel grid, but its impact on the patient treatment plan.