When the Mask Fails the Minority: Margin-Ring Texture Pooling for Imbalanced Ultrasound Diagnosis

Many medical image classifiers first outline the organ or lesion with a segmentation model and then classify only what lies inside that outline. This "look only here" step stops the classifier from learning shortcuts from irrelevant tissue. We argue that it also carries a hidden cost that falls mostly on the rare class. In breast ultrasound, malignant lesions are the minority class, and they are also the lesions with irregular, spiculated, poorly defined margins. Those are exactly the boundaries a segmentation model is most likely to draw too tightly. If that holds, a masked classifier systematically discards the most diagnostic tissue of the cases it can least afford to miss. Its overall accuracy can look excellent while its sensitivity on malignancies quietly depends on segmentation quality.

The proposal has two parts. First, we audit whether segmentation error is correlated with class in public breast ultrasound data. Second, we replace "pool features inside the mask" with pooling over three zones: the lesion core, a ring straddling the predicted boundary, and the surrounding tissue. The ring is described with multi-scale local binary patterns. A local binary pattern (LBP) is a texture code that records, for each pixel, which neighbors are brighter than it. It is therefore unchanged by any monotonic change in brightness or gain. The LBP statistics also tell us where the ring should end: we extend the ring outward while the tissue just outside the predicted mask still has lesion-like texture. The central claim is that this design keeps malignant-class sensitivity stable when masks are truncated and when scanner gain changes. Standard mask-weighted pooling loses both.

What Exists and Where This Sits

Segmentation-gated diagnosis is well established, and it motivates this work. The Role of Pleura and Adipose in Lung Ultrasound AI shows that unmasked lung ultrasound models profile subcutaneous fat. Masking to the pleura and the region beneath it raises ROC AUC from 0.6553 to 0.7742. Mask-weighted pooling, which averages deep features only over masked pixels, restricts a CT triage model to lung parenchyma in Computed tomography–based COVID–19 triage through a deep neural network using mask–weighted global average pooling. Lesion-Centric Latent Phenotypes from Segmentation Encoders for Breast Ultrasound Interpretability applies the same pooling to segmentation-encoder features on breast ultrasound and reports a malignancy probing AUC of 0.982 with external testing on BUS-BRA.

None of these works asks whether mask errors are distributed evenly across classes. The research file names margin truncation and error cascading as known weaknesses of this family. Notably, the lesion-centric paper needed a separate rule-based gate on boundary descriptors to rescue "deceptive malignant" cases, which raised margin accuracy on discordant cases from 59.1% to 90.9%. We read that as indirect evidence that the margin is where masked deep features fall short.

Two lines of work constrain the design. First, LBP is not free. Training deep segmentation networks on texture-encoded input: application to neuroimaging of the developing neonatal brain found that a segmentation network trained only on 10-pixel-radius LBP maps comes within 1–4% Dice of raw intensity. Its 1-pixel version collapses on small structures (hippocampus Dice about 0.0019). Single-scale, fine-radius LBP is therefore ruled out, and we use multiple radii scaled to lesion size. Second, Comparison Study of Radiomics and Deep Learning-Based Methods for Thyroid Nodules Classification Using Ultrasound Images found that hand-crafted texture features fed to an SVM inside a cropped ROI lose to a pruned CNN (AUC 0.6371 vs. 0.7127). We therefore do not propose texture instead of deep features. We propose texture pooled over a region that deep mask pooling throws away, with deep features retained.

The imbalance literature supplies tools for the segmentation stage. Generalised Dice Overlap as a Deep Learning Loss Function for Highly Unbalanced Segmentations introduces a Dice loss reweighted by class volume. A Unified Loss for Handling Inter-Class and Intra-Class Imbalance in Medical Image Segmentation introduces Balance Loss, which mines hard background pixels and down-weights easy interior pixels, and reports kidney-tumor Dice of 66.8 vs. 59.8 for cross-entropy on KiTS19. Both papers judge success by pixel overlap. We ask a different question: does a boundary-focused loss reduce the class-correlated part of mask error that harms downstream diagnosis? Pixel Dice does not measure that.

Core Claim

In imbalanced lesion classification, segmentation masks fail more often and more severely on the minority class. Mask-gated classifiers therefore inherit a sensitivity penalty that aggregate AUC hides. We claim that pooling gain-invariant multi-scale LBP statistics over a ring placed by texture around the predicted boundary, alongside the usual core features, removes most of this penalty. We also claim it makes malignant sensitivity robust to monotonic gain shifts between scanners.

Research Program

The audit comes first because the rest depends on it. We train segmentation models on BUSI (437 benign, 210 malignant, 133 normal) and BUS-UCLM, using cross-entropy, Generalized Dice Loss and Balance Loss. For each class we then measure three things on held-out and BUS-BRA images: Dice, signed boundary error (inward truncation versus outward spill), and the fraction of the true margin band missed. We then test whether, under mask-weighted pooling, a malignant lesion's mask truncation predicts that it will be a false negative.

If malignant and benign masks fail at similar rates and truncation does not predict false negatives, the premise is wrong. In that case we report the audit as the finding and drop the method.

The model takes a predicted soft mask and defines three zones. The core is the mask eroded by a fraction of the equivalent lesion radius. The ring is a band of width $w$ centered on the boundary. The surround is a further band beyond the ring. In each zone we compute uniform LBP histograms at radii proportional to lesion size (roughly 3%, 8% and 15% of the equivalent radius) and center-symmetric LBP histograms, which compare opposite neighbors and are more compact. We concatenate these with deep features pooled over each zone by mask-weighted pooling.

The ring width is adaptive. Starting from the predicted boundary, we step outward in thin shells. We keep a shell in the ring while the chi-square distance between its LBP histogram and the core histogram stays below a threshold. The threshold is tuned on validation data only, and we cap the ring's growth. This texture test is the one place where LBP does more than describe tissue: it corrects masks that are too tight. The classifier uses class-balanced sampling and is otherwise standard.

Baselines

The naive reference is a whole-image ImageNet-pretrained CNN trained with class-balanced sampling and standard intensity and affine augmentation on the same splits. It shows how much any form of masking helps here.

The strongest existing method is mask-weighted pooling of segmentation-encoder features as in Lesion-Centric Latent Phenotypes. That paper reports a malignancy probing AUC of 0.982, against 0.852 for a standard CNN and 0.774 for radiomics. It is the most direct competitor because it uses the same masks, the same data family and the same external cohort. We will reimplement it on the same encoder, segmentation model, splits and hyperparameter search budget as our method. We will not quote the published number as if it were comparable, because our pipeline need not match theirs exactly.

Because that AUC leaves little headroom, "beating it" is defined in three ways. On BUS-BRA we require higher malignant sensitivity at the benign specificity the baseline reaches at its own operating point, with confidence intervals from patient-level bootstrap. Under controlled mask erosion, malignant sensitivity must degrade significantly more slowly. Under monotonic gain perturbations at test time, sensitivity must drop less. Matching AUC while winning on these would count as success. Losing AUC by more than the bootstrap interval would not.

We also run HoVer-Trans, an ROI-free transformer that encodes the layers of breast anatomy. It reports AUC 0.924 on its private GDPH&SYSUCC set, and no reported result was found on Lacuna for it on BUSI. We will report it on the public data with the same tuning budget.

For the imbalance side, we compare against the diffusion augmentation of Hybrid Diffusion Model for Breast Ultrasound Image Augmentation. It reports accuracy 0.924 (texture-token variant) and F1 0.912 on BUSI, and we apply it as a minority oversampler in front of the mask-pooling baseline.

Three ablations isolate the key ingredient. The first uses a fixed-width ring instead of the texture-adaptive ring. The second pools deep features over the ring instead of LBP. The third swaps segmentation losses (cross-entropy, Generalized Dice, Balance Loss) under an otherwise unchanged classifier.

Why not just…?

Why not just dilate the mask by a fixed margin? This is the cheapest fix and may recover most of the gain. A fixed dilation over-includes surrounding fat and glandular tissue on well-outlined benign lesions while under-including on badly truncated malignant ones, because truncation varies with how irregular the lesion is. The fixed-ring ablation, swept over widths with the best width chosen on validation data, tests this directly. If it matches the adaptive ring on malignant sensitivity, we will report that the adaptive ring is unnecessary.

Why not just pool CNN features over the ring instead of LBP? This is the real threat. Our only principled reason to prefer LBP is that it is exactly unchanged by monotonic gain curves, ignoring ties and interpolation, whereas learned features are not. The deep-ring ablation, evaluated with and without test-time gamma and gain perturbations and on the BUS-BRA scanner shift, settles it. If deep ring features match LBP under perturbation, LBP should be dropped from the claim, leaving the ring as the contribution. The radiomics-versus-CNN gap in the thyroid study makes this outcome plausible.

Why not just use intensity augmentation or histogram equalization for gain robustness? Augmentation covers only the gain curves sampled during training. The lung ultrasound study shows augmented end-to-end models still learning tissue-layer shortcuts (AUC 0.6553). Augmentation also does nothing for mask truncation, which is the primary failure mode here. All learned baselines receive the same augmentation, so the perturbation test measures what invariance by construction adds beyond it.

Why not just reweight the classes or oversample malignancies? Reweighting changes how errors are traded between classes but cannot restore margin tissue that the mask has deleted. The diffusion baseline adds synthetic minority cases. Its reported accuracy drop after image-to-image refinement (0.905 vs. 0.924) hints that generation smooths high-frequency texture. We will also compare LBP histograms of real and synthetic malignant margins, to check whether synthetic oversampling teaches the classifier margin texture that is not physical.

Decisive Checks

The audit is the first gate. Class-conditional boundary truncation must be significantly larger for malignant lesions and must predict false negatives under mask-weighted pooling.

The second check is a mask-degradation sweep. We erode and dilate ground-truth and predicted masks by controlled fractions of lesion radius and plot malignant sensitivity at fixed specificity for mask pooling, fixed ring and adaptive ring. The method is supported only if its curve is flatter on the erosion side.

The third check is test-time gain perturbation on BUS-BRA with randomly drawn monotonic curves. We compare the drop in sensitivity for LBP-ring and deep-ring pooling.

The last check asks whether boundary-focused segmentation losses shrink the class gap in truncation more than their pixel Dice gains would predict. A positive result would show that imbalance losses should be judged by their effect on the minority class downstream, not by overlap alone.