Report Measurements as Free Supervision for Scribble-Trained 3D Lesion Segmentation in Small Cohorts

Training a 3D tumor segmenter usually means a radiologist outlines the tumor on every slice of every scan, which takes hours per scan. A cheaper option is to draw a few rough strokes ("scribbles") inside and outside the tumor and train only on those voxels. Scribbles show where a lesion is, but they say little about where it ends, so the boundary is left for the network to guess. When only a few dozen scans are available, the guess is poor. Yet almost every clinical scan already comes with a radiology report that states, in plain numbers, how many lesions there are and how large they are. This proposal uses those reported sizes as a training-time constraint on scribble-supervised models. The scribbles say where a lesion is. The report says how big it is. Neither is needed when the model runs on new scans.

The claim is narrow and testable. In small cohorts (10 to 50 scans), the main error of scribble-trained lesion segmenters is getting the extent wrong, not missing the lesion. Report diameters, enforced as a soft volume-and-size penalty anchored on the scribbles, should remove much of that extent error without extra annotation. The result would be lower volume error and better boundary metrics than scribbles alone, at the same annotation cost. In this framing the clinical text is not a second input stream to fuse with the image. It is a source of labels that is available only during training. That choice also sidesteps the missing-modality problem in multimodal models, because a report is never required at inference.

What exists and what is missing

The closest prior work is Report Supervision (R-Super). It uses a 70B-parameter language model to extract lesion count and diameters from CT reports. It then trains with a "volume loss", which matches the predicted voxel volume to an ellipsoid built from the reported diameters, and a "ball loss", which slides a sphere of the reported diameter over the probability map to place lesions. R-Super supplies our loss design. It differs from this proposal in three ways. It trains on more than 40,000 scans. It needs precomputed organ masks to limit where its losses search. It uses no human spatial cues. Our proposal works with tens of scans and replaces the organ mask with scribbles as the spatial anchor. This tests whether report constraints still help when the large dataset that made R-Super work is absent.

The scribble side is constrained by a negative result. Revisiting 3D Medical Scribble Supervision and Embarrassingly Simple Scribble Supervision showed that elaborate scribble methods tuned on cardiac MRI collapse elsewhere. ShapePU falls from 0.850 Dice to 0.369 across their ScribbleBench suite. Meanwhile a plain nnU-Net trained with a partial loss (cross-entropy and Dice computed only on scribbled voxels) reaches 0.813 mean Dice, against 0.856 for full masks. This rules out building on complex pseudo-labeling machinery, and it makes nnU-Net with partial loss the base we add to. Consistency methods such as TDNet and the affinity graphs of Con2R regularize boundaries using image cues only. Those cues are exactly what is weak for low-contrast lesions. Point-based PA-Seg relies on intensity gradients to stop expansion, with the same weakness. None of the scribble papers characterizes performance below about 50 scans.

On the small-sample side, CRLN reaches 91.74% Dice on Left Atrium with 8 fully labeled scans plus unlabeled data. It therefore represents the strongest alternative way to spend the same annotation budget: a few full masks instead of many scribbled scans. Multimodal small-sample work such as MFM-Geom and the label-efficient 2D TextDiff feeds clinical text into the network as an input. Both motivate the idea that clinical context carries information an image-only model lacks. Neither turns that context into a geometric constraint on dense 3D masks. MFM-Geom only classifies scans, and TextDiff is 2D only.

Core mechanism

Each training scan $i$ has a set of scribbles and a reported list of lesions with diameters. Each scribbled lesion is matched to a reported lesion by location and size order. For lesion $k$ we form a search region $\Omega_{ik}$: the scribbled foreground dilated by a margin equal to the reported major diameter. This region plays the role that R-Super's organ mask plays. The training objective is

$$ \mathcal{L} = \mathcal{L}_{pL} + \lambda \sum_{k} \max\!\Big(0,\ \tfrac{|\hat V_{ik} - V_{ik}|}{V_{ik}} - \epsilon\Big) + \mu\, \mathcal{L}_{ball}, $$

where $\mathcal{L}_{pL}$ is the partial cross-entropy plus Dice on scribbled voxels and $\hat V_{ik}$ is the summed predicted foreground probability inside $\Omega_{ik}$ times voxel volume. $V_{ik}$ is the ellipsoid volume implied by the reported diameters. $\epsilon$ is a tolerance reflecting radiologist measurement variability. $\mathcal{L}_{ball}$ is R-Super's sphere-matching term restricted to $\Omega_{ik}$. $\lambda$ and $\mu$ are loss weights.

Reported lesions with no scribble enter only through a count penalty over the organ region that the scribbles already outline. The tolerance $\epsilon$ matters. Report diameters are noisy, and a hard constraint would teach the model those errors. The model at inference sees only the image.

Research Program

The work has three parts. First, a controlled study on the lesion tasks in ScribbleBench (KiTS2023 kidney tumors, LiTS liver tumors, BraTS2020 brain tumors) using its simulated scribbles. "Reports" are synthesized from the ground-truth masks as rounded major and minor diameters plus count. Multiplicative noise is swept from zero up to levels beyond typical inter-reader variation. Training cohorts are subsampled to 10, 25 and 50 scans, with several random draws each, and evaluated on the full held-out test sets. Second, a diagnosis of the error that the method claims to fix. For each baseline we split segmentation error into missed lesions, false lesions, and extent error on detected lesions, and check whether extent dominates at small N.

Third, a transfer to real reports. We would use a subset of the report-paired CT sources that R-Super trained on, provided that subset includes voxel masks for evaluation. Diameters would be extracted with the large language model setup R-Super found necessary, since that paper reports that small extractors such as Qwen 0.6B fail. If no masked subset with real reports is accessible, the real-report claim is left untested and stated as such, not inferred from simulation.

Baselines

The naive reference is nnU-Net with partial loss on scribbles alone (Embarrassingly Simple Scribble Supervision). It is also the strongest existing scribble method, at 0.813 mean Dice over ScribbleBench at full dataset size. No reported result on Lacuna was found for it at 10 to 50 training scans, so we establish that number ourselves. The proposal is the same nnU-Net configuration, data splits, scribbles, epochs and augmentation, with only the extra report terms. A fair win means lower relative absolute volume error and better boundary distance (normalized surface Dice, HD95) at equal or better Dice, averaged over cohort draws, with confidence intervals that exclude zero.

The upper reference is fully supervised nnU-Net on the same N, which gives the 0.856 ceiling at full size. The annotation-matched competitor is CRLN (CRLN). It is given full masks for the number of scans whose total annotation time equals our scribble budget, with the rest left unlabeled. On Left Atrium it reports 91.74% Dice with 8 labeled scans, but no reported result on Lacuna was found for it on KiTS, LiTS or BraTS lesions at matched budgets.

TDNet (TDNet), which reports 85.74% Dice on WORD organs with scribbles, is included as a strong image-only boundary regularizer under identical tuning budgets.

Two ablations isolate the key ingredient. The first is an R-Super-style variant that uses report losses inside an organ mask without scribbles, to test what the scribbles add. The second removes the ball term so that only volume matching remains. A further control feeds the true diameters as an extra input channel at both training and test time. It shows whether using sizes as an input beats using them as a constraint when sizes happen to be available.

Why not just…?

Why not just threshold the scribble model's output so each lesion's volume matches the report, then retrain on those pseudo-labels? This is the most serious threat, because it uses the same information in two simple steps. It will be run as a full baseline: train nnU-Net with partial loss, choose a per-lesion threshold that matches the reported volume, then retrain on the result. If it matches the end-to-end loss within noise, we adopt it and report that the value lies in the report signal, not in the loss. Our expectation is that it fails where the first-stage boundary has the wrong shape, not just the wrong scale. A threshold can only grow or shrink a wrong contour, whereas a training-time penalty changes the features the model learns. The per-lesion error analysis will show whether this happens.

Why not just spend the annotation time on a few full masks and use semi-supervised learning? CRLN shows that very few full masks can go far on high-contrast targets. It is the matched-budget baseline above. If it wins on lesions too, the conclusion is that scribbles plus reports are not worth it, and the note will say so.

Why not just give the clinical data to the network as input, as multimodal fusion work does? Inputs require the report at inference, which breaks in exactly the settings where records are incomplete. With tens of scans, a learned fusion module also adds parameters that can overfit. The input-channel control measures how much is lost by restricting reports to training time.

Decisive checks

The idea survives only if four things hold. At 10 to 25 scans, the report terms must cut relative volume error and improve surface metrics over nnU-Net with partial loss on at least two of the three lesion tasks. The gain must persist at realistic measurement noise and shrink gracefully as noise grows. The gain must be larger at small N than at full dataset size, as expected if reports substitute for the shape knowledge that more data would otherwise provide. And it must beat the threshold-and-retrain baseline. If the error breakdown shows that small-N failures are mostly missed lesions rather than wrong extent, the premise is wrong and size constraints are the wrong fix.