Shape-Faithful Segmentation: Training Lesion Masks to Get the Clinically Used Geometry Right

Radiologists grade many lesions by their shape. A thyroid nodule earns risk points for being taller than it is wide or for having a lobulated margin. A kidney mass looks more malignant when it is flat rather than round. A tumor pressing a vein shaped like a teardrop signals invasion. Automated pipelines usually produce these judgments in two steps: a network draws a mask, then a separate program computes shape numbers from it. The network, however, is trained to maximize pixel overlap. It has no reason to get the shape numbers right, and at faint, low-contrast edges it tends to smooth away exactly the bumps and elongations that carry diagnostic weight. Size makes this worse. The lesions with the most ambiguous edges are often the smallest, and there a one-pixel error barely changes overlap but can swing a shape ratio across a grading threshold.

We propose to train segmentation networks with a loss defined directly on a small set of dimensionless shape descriptors: axis ratios, compactness, and a size-relative measure of how much of the lesion sits in protrusions. These are the quantities clinical grading schemes turn into points. Because they do not depend on lesion size, they give scale invariance without the inverse-area weighting that blows up gradients on tiny, noisy lesions. We also propose a new way to score segmenters. Instead of asking only "how much do the masks overlap," we ask "how often does the segmentation error change the patient's grade."

Core Claim

A segmentation loss that penalizes error in dimensionless, clinically used shape descriptors has three properties together. First, it lowers the rate at which boundary errors flip grading-relevant shape categories, compared with overlap and boundary-distance losses that reach similar Dice and Hausdorff scores. Second, it is invariant to lesion size by construction. Third, it can be trained from cheap categorical feature labels such as "taller-than-wide: yes/no" when pixel masks are scarce.

The concrete test sits in thyroid ultrasound under the ACR TI-RADS scheme (the American College of Radiology's five-tier thyroid risk system), where the shape and margin criteria are explicitly geometric. A secondary test uses prostate MRI to check that the idea transfers to 3D.

Relation to Prior Work

The motivation comes from radiomics studies showing that shape and boundary descriptors, not bulk intensity, carry the signal. Some of the evidence concerns malignancy and outcome. Wu et al. found 3D flatness to be the dominant malignancy feature in renal tumors, and the shell-feature work showed that a thin boundary layer predicts distant failure better than whole-volume texture. The pancreatic confluence study found that focal contour irregularity reached 78.7% specificity for vein invasion, against 38.3% for the 180-degree contact rule. Other evidence concerns intermediate risk tiers: the PI-RADS 3 study found that shape-inclusive radiomics trained on intermediate-grade cases generalized best. All of these studies relied on expert manual masks. None of them asks what happens to the descriptors when the mask comes from a network, and closing that gap is the purpose of this proposal.

The boundary-focused losses constrain and inform the design. Boundary loss integrates softmax outputs against a distance map to the true contour. Boundary Difference over Union adapts its boundary emphasis using the ratio of perimeter to area. Structure boundary preserving segmentation uses key points and an adversarial shape critic. All three penalize contour deviation more or less uniformly, or judge plausibility in general. None targets the specific ratios a grading scheme reads. A mask can have good Hausdorff distance and still round off the one lobulation that earns two TI-RADS points.

SIOpt achieves size invariance by computing losses per connected component and averaging them. It was built for high-contrast natural images, and inverse-size weighting is precisely what amplifies noise on small, ambiguous lesions. We obtain invariance through dimensionless targets instead, and we switch the shape loss off when a lesion falls below the resolution at which its shape can be identified.

On the grading side, CMCNet aligns cropped-image embeddings with text descriptions of TI-RADS features and treats "shape" and "margin" as categorical tokens. The PI-RADS MLLM work similarly produces qualitative descriptor words. MFM-Geom uses "geometry" in the sense of the covariance manifold of features, not the lesion's physical contour. These methods are the strongest direct graders, and they supply the comparison point. They never produce a measurable contour, so their shape judgments cannot be checked against a caliper-style measurement.

Research Program

The first component is a set of differentiable descriptors computed from a soft mask $S_\theta$, meaning the network's per-pixel foreground probabilities. We restrict ourselves to descriptors a clinician would recognize.

Elongation and orientation come from the eigenvalues of the probability-weighted coordinate covariance. This yields the anteroposterior-to-transverse extent ratio behind "taller-than-wide," and flatness in 3D.

Compactness is squared perimeter over area, with perimeter estimated by the total variation of the soft mask.

Irregularity is defined as

$$\rho(S) = 1 - \frac{|\mathrm{open}_{r}(S)|}{|S|}, \qquad r = c\sqrt{|S|},$$

where $|S|$ is the soft area, $\mathrm{open}_r$ is a soft morphological opening (erosion then dilation, implemented with min- and max-pooling) with structuring radius $r$, and $c$ is a fixed fraction. Setting the radius proportional to the lesion's own linear size makes $\rho$ the fraction of the lesion lying in protrusions narrower than a fixed proportion of its size. A magnified copy of a lesion therefore gets the same $\rho$. This makes $\rho$ a size-relative spiculation measure rather than a count of noisy pixels.

The training objective is

$$\mathcal{L} = \mathcal{L}_{\text{Dice}} + \lambda \sum_k \mathbb{1}[\,r \ge r_{\min}\,]\; w_k \,\big| d_k(S_\theta) - d_k(G) \big|,$$

with the following terms. $G$ is the reference mask. The $d_k$ are the dimensionless descriptors above. The weights $w_k$ normalize each descriptor by its spread across the training set. The indicator switches the shape term off when the structuring radius falls below $r_{\min}$, a floor tied to the scanner's physical resolution. The Dice term stays in place for the same reason boundary loss needs a regional companion: shape terms carry no signal before the network predicts any foreground.

The resolution floor is the direct answer to the file's gap on gradient instability. Below it, we treat the lesion's shape as unidentifiable and do not ask the network to fit it.

The second component is weak geometric supervision. The STN thyroid dataset provides bounding boxes and expert labels for the TI-RADS shape and margin categories. The file does not indicate that it provides pixel masks, so we will annotate masks for a subset. For the remaining nodules, a small logistic head maps the predicted mask's descriptors to the categorical labels, for example the extent ratio to taller-than-wide and $\rho$ plus compactness to smooth versus lobulated/irregular. The box constrains where the mask can lie.

This mode has an obvious failure: the network could distort the mask to satisfy the label. We will detect that by checking descriptor error and Dice on held-out fully masked cases.

The third component is the evaluation protocol that ties boundaries to diagnostic consequences. For every test lesion we compute TI-RADS shape and margin points from the predicted mask and from the reference mask. We report the grade-flip rate, the fraction of lesions whose points or final tier change. We stratify it by lesion size and by ground-truth tier, since the intermediate tiers are where discordance is highest.

Baselines

The naive reference is a U-Net trained with Dice plus cross-entropy and a multitask cross-entropy grading head. This is the setup the file reports CMCNet beating through its multitask comparison. All segmenters share one backbone, the same training data (including the same masked subset), and the same number of hyperparameter trials, with augmentation fixed across methods.

The strongest segmentation competitors are Boundary loss combined with Generalized Dice and Boundary DoU. Boundary loss is reported at ISLES Dice 0.656, up from 0.575 for Generalized Dice alone (Kervadec et al.). Boundary DoU is reported at Synapse Hausdorff distance 26.29 mm versus 31.45 mm (Boundary DoU). They are the strongest competitors because they already concentrate supervision on the contour. SIOpt's size-invariant Dice is included as the representative component-wise, size-invariant loss.

Beating these baselines means a statistically lower grade-flip rate and lower descriptor error at Dice and Hausdorff within a small tolerance of the best competitor. The gain must hold in the smallest-lesion stratum. A method that wins on flip rate by sacrificing overlap does not count.

For grading, the strongest existing method is CMCNet, reported at 75.17% balanced accuracy and 91.78% weighted F1 for 5-class TI-RADS on STN (CMCNet). We will rerun it on our splits rather than quote the published number across different splits. Our grader feeds measured descriptors, alongside image features, into the same classifier.

Three ablations isolate the key ingredients. The first replaces dimensionless descriptors with dimensional ones such as raw perimeter and area, to test whether size invariance matters. The second replaces the resolution floor with inverse-area weighting, to test Gap 3 directly. The third removes the weak-label head, to measure how much the categorical labels contribute.

Why not just…?

Why not just grade directly from the image, as CMCNet does, and skip masks altogether? This is the most serious threat. If final-tier accuracy is the only goal, a direct grader may match or beat any mask-based route, and we will report honestly if it does. Our claim is narrower. A measured contour makes the shape and margin points auditable and reproducible, and descriptors should help most in intermediate tiers and small nodules, where categorical text anchors have the least to hold onto. The per-tier comparison against a retrained CMCNet settles this. If CMCNet plus our descriptors does not beat CMCNet alone on TR3 and TR4, the diagnostic half of the claim fails and only the segmentation-fidelity result stands.

Why not just run nnU-Net and then extract radiomic shape features offline? This decoupled pipeline is exactly what our flip-rate metric evaluates. The file's evidence is suggestive rather than decisive. Wu et al. validated flatness only on manual masks, and the pancreatic study shows that the informative signal is focal contour deformation, which overlap-trained networks smooth away. We run the decoupled pipeline as a baseline. If its flip rate matches ours at equal Dice, the descriptor loss is unnecessary.

Why not just use Boundary loss or Boundary DoU, which already target the contour? These losses reduce average contour distance, but a small average error can sit entirely on one lobulation. We will test this directly by comparing descriptor error and flip rate among methods matched on Hausdorff distance. If boundary losses give the same descriptor fidelity once Hausdorff distance is matched, that finding refutes our premise.

Why not just smooth or ensemble the masks at test time to stabilize shape numbers? Smoothing stabilizes descriptors by biasing them toward roundness, which erases the irregularity that earns margin points. We include a test-time-augmentation ensemble with mask smoothing as an extra baseline and report its bias on lesions labeled irregular.

Proposed Experiments

The primary experiment uses STN with 5-fold patient-level splits and a mask-annotated subset whose size we will fix after an annotation pilot. It reports descriptor error, TI-RADS shape and margin flip rate, and Dice and Hausdorff distance, all stratified by nodule size. It also reports 5-class balanced accuracy and weighted F1, with per-tier recall.

The first decisive check is the matched-overlap comparison: among models within one Dice point of each other, does the descriptor loss cut flip rate? The second is the size stratum. In the smallest third of nodules, the resolution-floor variant should be more stable across training seeds than the inverse-area variant, measured by variance of descriptor error over five seeds.

For 3D transfer we use PI-CAI lesion annotations to compute flatness and irregularity flip rates. We test whether descriptors added to an image classifier improve detection of clinically significant cancer in the low-data regime. The reference point there is MFM-Geom's AUC-PR of 90.67 with 10% of the training data (MFM-Geom), rerun on our splits. PROSTATE158 serves as an external check.

As a segmentation sanity check outside grading, we evaluate on ISLES and WMH, where the Boundary-loss numbers above give a direct reference. We also evaluate on ETIS small polyps, reported at CaraNet mDice 74.7% (CaraNet), to confirm the shape term does not cost overlap on small, low-contrast targets.

A negative result on the matched-overlap check would itself be informative. It would mean overlap-trained masks already preserve clinically used geometry, and that the radiomics literature's reliance on manual masks is more caution than necessity.