Many interpretable diagnostic systems justify a "malignant" call by pointing to the lesion's border: it is irregular, spiculated, or lobulated. Increasingly, that border description is computed from a contour that a segmentation network drew. This proposal tests a specific failure mode. Segmenters are trained on data dominated by benign cases and scored with area-overlap metrics such as Dice, which barely register errors along the edge. If such a segmenter systematically smooths the ragged edges of the rare malignant lesions, then the border explanation is weakest on exactly the cases where clinicians most need it, and none of the standard numbers would reveal this.
We propose a diagnosis-stratified boundary audit with two parts. The first scores contours with boundary-focused metrics separately for benign and malignant lesions. The second checks each predicted border descriptor against the spread of values that multiple human annotators produce for the same lesion. The audit is then used to compare cheap and targeted fixes for the class gap it exposes. The contribution is mainly a measurement and a finding: how much of a model's border explanation can be trusted, for which class, and which training change restores it. A method is secondary.
Core Claim
The claim is that contour-derived border concepts are biased toward "regular" on minority malignant lesions. We expect this bias to be visible under boundary-restricted metrics and invisible under Dice, and to exceed what human disagreement alone would explain.
Three findings in the file make this plausible. Boundary IoU restricts overlap to a band of width $d$ around both contours. It shows that area metrics hide edge errors: on COCO, Mask R-CNN's large-object score falls from 66.0 Mask AP to 25.9 Boundary AP. The IMA++ inter-annotator study shows that experts already disagree more on malignant borders, with mean pairwise Dice of 0.753 versus 0.791 for benign lesions ($p<0.001$). Those borders are therefore intrinsically harder. The Balance Loss paper shows that boundary pixels are the hard minority even inside the foreground. Together these motivate the gap.
The closest constructive work is Lesion-Centric Latent Phenotypes. It computes margin sharpness and compactness from predicted breast-ultrasound contours and uses them in a symbolic risk-escalation rule, raising edge-case margin accuracy from 59.1% to 90.9%. However, it never measures whether those contours are accurate at the boundary, let alone per diagnostic class. Our work supplies that missing check.
Research Program
Audit design. For each image we have a predicted mask and the masks of several annotators. We compute three scores, stratified by diagnosis:
- Dice.
- The guarded boundary score $\min(\text{Mask IoU}, \text{Boundary IoU})$. Cheng et al. recommend this to block the "hollow ring" loophole, in which a prediction covers only the edge band and omits the interior.
- Normalized Surface Distance, the fraction of contour points lying within a tolerance of the reference. Duque et al. found it tracks boundary failures that Dice misses.
Because the band width $d$ is a known sensitivity, we report results across a sweep of $d$ in physical units rather than a single heuristic setting. The model's malignant-minus-benign gap is always reported next to the same gap computed between human annotators. This tells us how much of the model's deficit exceeds intrinsic ambiguity.
Concept fidelity. Next we measure how contour error turns into concept error. For each lesion we compute simple, reproducible border descriptors from a contour:
- a convexity deficit, which is one minus the lesion area divided by its convex hull area;
- a curvature-based roughness score.
These descriptors are computed from the predicted contour and from each annotator's contour. Let $a_i^{\min}$ and $a_i^{\max}$ be the smallest and largest descriptor values across annotators for lesion $i$, and let $\hat a_i$ be the value from the predicted contour. Define the signed tolerance error
$$ e_i = \begin{cases} \hat a_i - a_i^{\max} & \text{if } \hat a_i > a_i^{\max} \\ \hat a_i - a_i^{\min} & \text{if } \hat a_i < a_i^{\min} \\ 0 & \text{otherwise.} \end{cases} $$
An explanation counts as unfaithful only when it falls outside the range of competent human readings. A negative $e_i$ means the model smoothed the border.
This treats annotator disagreement as a tolerance on the explanation, following the IMA++ argument that disagreement is signal rather than noise. Unlike that paper, we do not compress disagreement into a single predicted scalar. The headline quantities are the fraction of lesions with $e_i \neq 0$ and the mean sign of $e_i$, both per class.
Interventions. We compare training changes for the segmenter that might close a class gap. Each targets a different hypothesis about its cause:
- Diagnosis-balanced sampling assumes too few malignant images.
- Balance Loss assumes the problem is pixel-level background and boundary imbalance.
- Diagnosis-weighted boundary-band loss assumes it is the joint scarcity of malignant edge pixels. This loss upweights pixel loss inside the band of width $d$ around the reference contour by the inverse frequency of the image's diagnosis.
The third option is the one ingredient we add. We keep it only if it beats the two simpler options on the audit.
Downstream diagnosis. Finally we feed the descriptors into a small concept-bottleneck classifier, a model forced to predict the diagnosis only through named, human-readable features (Koh et al.). This shows whether repairing minority-class contours changes melanoma sensitivity and balanced accuracy.
Baselines
Naive reference point. A U-Net trained with Dice plus cross-entropy, with no rebalancing, evaluated only by Dice. This is the de facto pipeline behind contour-derived phenotypes. All audit numbers are reported relative to it.
Strongest segmentation competitor. Balance Loss from Xu et al.. It is the only method in the file that explicitly targets both background-versus-lesion imbalance and boundary-pixel difficulty. It reports a kidney-tumor Dice of 66.8 on KiTS19 against 59.8 for cross-entropy. We run it with its two-stage schedule on the same backbone, the same splits and the same epoch and tuning budget as every other arm. No reported Boundary IoU or class-stratified boundary result exists in the file for this or any medical model; no reported result was found on Lacuna for Boundary IoU on IMA++, BUSI or LIDC-IDRI. Beating it means a smaller malignant-benign gap in the guarded boundary score and fewer out-of-tolerance malignant descriptors, with benign Dice not lower by more than one point.
Simple rebalancing competitor. Diagnosis-balanced sampling under the same budget.
Direct concept prediction. For the downstream claim, the strongest competitor is the multi-task model of Abhishek et al. It reports a 4.2% balanced-accuracy gain across five dermoscopy datasets. We retrain it on our splits. We also include a concept-bottleneck classifier whose border concept is predicted directly from pixels rather than measured from a contour. On ultrasound we compare against the lesion-centric phenotype pipeline. It reports 0.982 malignancy AUC on BUS-BRA, so the bar there is not lowering that AUC while reducing the out-of-tolerance rate.
Ablation. The diagnosis-weighted band loss is compared with an identical band loss that has uniform class weights. This isolates whether the diagnosis weighting matters or whether simply emphasizing the boundary suffices.
Why not just…?
Why not just report Dice, since it correlates with everything else? Dice is dominated by interior pixels, so a smoothed melanoma border can still score well. Cheng et al.'s 66.0 versus 25.9 gap and Duque et al.'s NSD sensitivity are direct evidence. Our decisive check settles this per image. If Dice predicts descriptor error $|e_i|$ as well as the guarded boundary score does, the boundary part of the audit is unnecessary, and we will say so.
Why not just oversample malignant images? This is a real threat. If the smoothing bias comes from too few malignant examples, balanced sampling may close the gap at no cost. Tousignant et al. warn that oversampling encourages memorization, but that warning concerns classification rather than contours. We therefore run sampling as a full arm. If it matches the band loss, the method contribution is dropped and the paper reports the audit and the finding that sampling suffices.
Why not just use Balance Loss? Balance Loss reweights pixels by difficulty and background count. It is blind to which diagnosis an image carries, so it may still leave rare-class edges under-trained. The uniform-weight band ablation and the Balance Loss arm together show whether pixel-level balancing already removes the class-conditional gap.
Why not just predict "irregular border" directly from the image and skip contours? Direct concept heads avoid segmentation error. But they offer no geometric object that a clinician can inspect. Veltmeijer et al. report that forcing ABC-rule concepts hurt accuracy on benign nevi, while auxiliary segmentation helped melanoma detection. We include the direct head as a baseline. It is scored against the same annotator tolerance by thresholding its output against the annotator-derived range, so the comparison is on equal terms.
Proposed Experiments
The primary dataset is IMA++: 2,394 dermoscopy images with 5,111 masks from 15 annotators, plus diagnosis labels. It is the only resource in the file with both multi-annotator contours and diagnoses. ISIC 2017 adds single-annotator replication for the downstream classifier. For ultrasound we train on BUSI and BUS-UCLM and test on BUS-BRA, mirroring the lesion-centric phenotype protocol. That way the margin descriptors driving its symbolic gate can be audited directly. There, the per-lesion tolerance interval is unavailable, so we report the boundary-metric class gap only. LIDC-IDRI tests generality in 3D CT. We deliberately include the non-consensus nodules that PN-SAMP excluded, because those ambiguous borders are the regime of interest.
There are three decisive checks:
- Existence of the bias. On the unbalanced baseline, the malignant-benign gap in the guarded boundary score should exceed the corresponding human-annotator gap. Malignant tolerance errors should be predominantly negative, meaning smoothed borders. If neither holds, the bias does not exist and the proposal stops there.
- Which metric predicts explanation error. Across images, the guarded boundary score should predict $|e_i|$ substantially better than Dice.
- Which fix works. Among the interventions, the winner is whichever most reduces the malignant out-of-tolerance rate at matched benign accuracy. We then check whether it raises melanoma sensitivity in the concept-bottleneck classifier.
Results are reported with bootstrap intervals over images and across the $d$ sweep, so that no conclusion rests on a single band width.
References
- Boundary IoU: Improving Object-Centric Image Segmentation Evaluation
- What Can We Learn from Inter-Annotator Variability in Skin Lesion Segmentation?
- A Unified Loss for Handling Inter-Class and Intra-Class Imbalance in Medical Image Segmentation
- Lesion-Centric Latent Phenotypes from Segmentation Encoders for Breast Ultrasound Interpretability
- Ultrasound segmentation analysis via distinct and completed anatomical borders
- Concept Bottleneck Models
- Deep Learning in Medical Image Analysis
- Integrating Clinically-Relevant Features into Skin Lesion Classification
- Joint learning for pulmonary nodule segmentation, attributes and malignancy prediction