Region-Normalized Hard Negative Mining for Small Stroke Lesions

When a lesion covers less than one percent of a brain scan, segmentation networks are commonly trained with "hard negative mining": the loss keeps only the healthy voxels that the network currently scores as most lesion-like and weights them against the lesion voxels. The problem is that healthy tissue is not equally noisy everywhere. Some anatomical regions routinely look ambiguous, such as tissue next to fluid-filled spaces or in deep grey matter. Others are quiet. If the loss ranks healthy voxels by raw score, its fixed budget of negatives piles up in the noisy regions. The network then learns to suppress everything there, including faint small lesions that happen to sit in those regions.

We propose ranking each healthy voxel by how unusual its score is relative to other healthy voxels in the same anatomical region, not by its raw score. This is a one-line change to the mining rule. It uses a standard brain parcellation and running statistics of the network's own outputs, and needs no extra labels. The claim is specific and testable. On multi-site stroke MRI, region-normalized mining should raise recall of small lesions in high-variance regions without adding false positives elsewhere. We also check whether it beats the obvious cheaper fix of simply choosing a different decision threshold per region after training.

Core Claim

A Unified Loss for Handling Inter-Class and Intra-Class Imbalance in Medical Image Segmentation introduced Balance Loss. It supplies the mechanism we modify. Its inter-class term selects exactly $f$ background voxels, where $f$ is the number of foreground voxels, choosing the ones with the highest predicted lesion probability. It gives them the same total weight as the foreground. This counters the pile-up of tiny soft errors over a huge background, and it lifts kidney-tumor Dice on KiTS19 from 59.8 (cross-entropy) to 66.8.

The selection criterion depends only on the prediction. Regional variability research gives reason to think the prediction is spatially confounded. Characterizing patterns of diffusion tensor imaging variance in aging brains fit mixed-effects models on 2,843 healthy subjects. It showed that variance in diffusion measures is region-specific and is modulated by sex, scan interval, and motion in region-dependent directions. Scalable group level probabilistic sparse factor analysis estimated noise precision voxel by voxel in fMRI and found it spatially structured rather than uniform.

Neither study uses T1-weighted stroke MRI, the modality we work with. Their role is to motivate a hypothesis, not to supply variance maps we could transfer. Our hypothesis is that in T1 stroke segmentation, the network's background scores also have region-dependent spread. Raw-score mining then concentrates the $f$ negatives in a small set of high-spread regions.

Our replacement rule scores each background voxel $i$ in parcellation region $r(i)$ with the quantity below. Here $s_i$ is the network's lesion logit. The terms $m_r$ and $\sigma_r$ are the mean and standard deviation of background logits in region $r$, tracked as exponential moving averages over recent training batches.

$$h_i = \frac{s_i - m_{r(i)}}{\sigma_{r(i)} + \epsilon}$$

The top-$f$ voxels by $h_i$ are mined in place of the top-$f$ by $s_i$. The rest of Balance Loss is unchanged, including its intra-class term and its weighting.

Research Program

The first stage is diagnostic, and the rest of the program depends on it. We train SPiN from Small Lesion Segmentation in Brain MRIs with Subpixel Embedding with Balance Loss on ATLAS v2.0. SPiN is a U-Net with a parallel branch that works at twice the input resolution so that tiny lesions survive downsampling. ATLAS v2.0, from A Large, Curated, Open-Source Stroke Neuroimaging Dataset to Improve Lesion Segmentation Algorithms, has 1,271 T1 scans from 44 cohorts, all registered to a common template.

We parcellate every scan into the 132 SLANT regions that Gao et al. use and log which region each mined negative falls in. We then test two things. The first is whether mined negatives concentrate in a few regions, measured as the entropy of the per-region mining distribution against the region-volume distribution. The second is whether per-region small-lesion recall is lower where mining concentration is higher, after controlling for lesion frequency. Liew et al. show that lesion frequency is itself spatially structured along middle cerebral artery territory, so this control is essential.

If mining is not concentrated, or concentration is unrelated to recall, the hypothesis is false and we report that. Region-normalized mining would then have no mechanism to act on.

If the diagnostic is positive, the second stage compares the mining rules under identical training. The third stage asks whether any gain comes from anatomy or merely from spreading negatives out. It also tests whether the gain transfers to the held-out cohorts, where scanner-dependent variance differs from training.

Baselines

The naive reference point is SPiN trained with plain binary cross-entropy. Wong et al. report this as their best loss on ATLAS v1.2, with DSC 0.703 overall and 0.424 on lesions under 100 pixels on their 212/27 split. We rerun it on that same split to confirm we reproduce those figures before trusting any comparison.

The strongest existing method for this question is SPiN trained with full Balance Loss, using its published two-stage schedule and $\alpha = 0.5$. It is the only method in the file that explicitly mines hard background voxels. No reported result on Lacuna exists for Balance Loss on ATLAS, and none exists for any method on ATLAS v2.0, so we produce both numbers ourselves. All variants share the same data splits and augmentation, the same number of GPU-hours, and the same search budget of eight learning-rate and loss-weight configurations, selected on a validation fold carved from the 655 training scans.

Region-normalized mining counts as beating Balance Loss under one condition. Its small-lesion Dice and lesion-wise recall must be higher on the hidden 300-scan test set and the 316-scan generalizability set, averaged over three seeds, with confidence intervals that exclude zero. Its false positives per scan must not be worse.

Two ablations isolate the key ingredient. The first, a quota ablation, mines raw-score negatives but caps each region's share in proportion to its volume. It tests whether merely spreading negatives is enough. The second, a shuffled-atlas ablation, applies our rule with region labels randomly permuted across spatially contiguous blocks. It tests whether anatomically meaningful regions matter or any spatial grouping works.

Why not just…?

Why not just pick a separate decision threshold for each region after training? This is the most serious threat. It costs nothing and directly equalizes per-region false positive rates. What it cannot do is change the features the network learned. If raw-score mining taught the network to flatten lesion scores in noisy regions, no threshold can recover lesions whose scores sit inside the healthy distribution.

We settle this by comparing region-normalized mining against Balance Loss plus per-region thresholds calibrated on validation data. The comparison uses per-region lesion-level recall at matched false positives per region, a free-response operating curve. If post-hoc thresholds match our method, the training change is unnecessary, and we will say so and publish the calibration procedure instead.

Why not just give the network its location, as coordinate channels or the ATLAS lesion-frequency map? A location prior tells the network where lesions usually are. It does not change how the loss budget is spent among healthy voxels. It may even worsen the problem, by further lowering scores for rare-location lesions.

We include SPiN plus Balance Loss plus three coordinate channels and the population lesion heatmap as a baseline. We report its small-lesion recall stratified by lesion-frequency tertile. We expect its gains to fall in high-frequency territory and ours in high-variance regions.

Why not just let the network predict its own per-voxel noise level and down-weight noisy voxels? A learned uncertainty head can absorb region-dependent noise, but it also learns to call real small lesions "noisy" and discount them. Nothing in the file shows that this avoids that failure. We run this variant: a second output predicts a per-voxel log-variance that scales the cross-entropy term. We compare its small-lesion recall to ours.

Why not just use unsupervised normative restoration? Unsupervised Lesion Detection via Image Restoration with a Normative Prior trains a generative model on healthy scans and flags the residual between a scan and its "healed" version. It reaches AUC 0.83 on BRATS 2017. Its authors report failure on small lesions, however, and it applies a uniform threshold to the residual. It is therefore a weaker competitor for small supervised targets. We do not rerun it as a main baseline.

Proposed Experiments

The primary benchmark is ATLAS v2.0, reporting results on the hidden test set and, separately, on the hidden-cohort generalizability set. Metrics are overall Dice, Dice on lesions under 100 voxels, lesion-wise recall, and false positives per scan. Every metric is also broken down by SLANT region, grouped into tertiles of measured background-score spread. ATLAS v1.2 with the Wong et al. split is a reproduction anchor, so that our SPiN numbers can be checked against the published 0.703 and 0.424.

To test whether the idea depends on brain anatomy, we run a secondary experiment on KiTS19. There, "regions" are coarse supervoxel clusters of the CT, since no parcellation exists. The reference is Balance Loss's reported 66.8 Dice on the kidney-tumor task.

Three checks decide the outcome. The first is the diagnostic: raw-score mining must be spatially concentrated, and that concentration must predict lower small-lesion recall. The second is that region-normalized mining must beat the per-region-threshold baseline at matched regional false positive rates, not merely beat Balance Loss. The third is that the shuffled-atlas ablation must lose most of the gain. If the first check fails, the premise is wrong. If the second fails, the contribution reduces to a calibration recipe. If the third fails, the benefit comes from spreading negatives out rather than from regional variability, and we would report it as such.