When doctors annotate medical images cheaply, by drawing a few quick strokes ("scribbles") inside each structure instead of tracing every boundary, the usual recipe is self-training. A segmentation network guesses a full mask, the guess is cleaned up, and the network is retrained on its own cleaned guesses. The weak point is the cleanup. A high-capacity network is very good at memorizing its own mistakes, so errors near boundaries get locked in and accuracy stalls or falls after a couple of rounds. We propose replacing the cleanup step with a small, separate classifier. For each image, it is refit from scratch in every round on the expert's original strokes. It then decides the uncertain voxels near boundaries. The classifier is a support vector machine (SVM) with a radial basis function (RBF) kernel, which separates classes using a Gaussian similarity between feature vectors. Its bandwidth knob, $\gamma$, moves it smoothly from almost linear to almost nearest-neighbor. That makes it a natural instrument for the question we actually care about: how much capacity should the label-correcting model have, given how few labeled voxels the expert supplied?
The opportunity is concrete. In rare-disease and small-cohort settings, expert time is the binding constraint. Here the failure of self-training is documented: a recursive U-Net cyst segmenter peaked at its second round and degraded at its third (Self-Learning to Detect and Segment Cysts in Lung CT Images without Manual Annotation, 75.87% then 74.94% Dice). Separately, a broad body of evidence shows that capacity mismatched to data size hurts more than it helps. If a capacity-matched corrector, anchored to the expert's strokes, stops that degradation, it gives a cheap, inspectable component that slots into any scribble pipeline.
Core Claim
Confirmation bias in scribble self-training comes from letting the same high-capacity model both propose and adjudicate labels. Decoupling the two helps. The adjudicator should be low-capacity, refit per image, and trained on expert strokes rather than on prior pseudo-labels. Doing this should keep accuracy from degrading across rounds and should close part of the gap to full supervision when only 5 to 20 training patients are available.
Our second, more falsifiable claim is that the best corrector capacity follows a predictable trend. The best kernel bandwidth should grow with the number of scribbled voxels per image, much as the model-complexity literature finds crossover points between simple and complex models as data grows. For example, Neural network language models for low resource languages mapped such a crossover. We would be the first, as far as this file shows, to measure that curve for the label corrector itself rather than the backbone.
What is new relative to the closest work is where the uncertainty is resolved and by what. Learning to Segment Medical Images with Scribble-Supervision Alone expands scribbles by random walk, refines with a dense conditional random field (CRF), and discards voxels the network itself is unsure about via Monte Carlo dropout. That means the network still judges its own labels. ScribbleVS uses a teacher network that is a moving average of the student, plus competitive pseudo-label selection. The teacher shares the student's architecture and biases. Neither uses an external model with explicitly bounded capacity, and neither re-anchors to the strokes each round.
Research Program
Each training image $i$ has scribbled voxels $S_i$ with class labels from the expert. In round $t$, the current segmenter produces a soft mask. Voxels are split into a confident set $P_i$ and an uncertain band $U_i$, roughly the voxels near predicted boundaries. For each image and each pair of adjacent structures, we fit a weighted RBF-SVM on per-voxel features $\phi(x)$:
$$\min_{w,b,\xi}\ \tfrac12\|w\|^2 + C\sum_{x\in S_i}\xi_x + C'\sum_{x\in P_i}\xi_x \quad \text{s.t. } y_x(w^\top\phi(x)+b)\ge 1-\xi_x,\ \xi_x\ge 0,$$
with kernel $K(x,x')=\exp(-\gamma\|\phi(x)-\phi(x')\|^2)$. Here $C$ is the penalty on violating expert strokes, $C' \ll C$ the much smaller penalty on violating the network's confident guesses, and $\xi_x$ the slack measuring how badly voxel $x$ violates the margin. The SVM does two jobs. First, it relabels every voxel in $U_i$. Second, it flags confident pseudo-labels with $\xi_x>1$, meaning the SVM classifies them the other way, and those are dropped from the next round's training set. The segmenter is then retrained on strokes, surviving confident labels, and SVM-decided band labels.
Because $C \gg C'$, the expert's strokes dominate the boundary every round. Information from earlier rounds enters only through low-weight, contestable points, never as hard labels.
The feature map $\phi$ is a deliberate design choice, because a corrector that sees the student's own deep features may inherit the student's bias. We will compare three options. The first is intensity, local texture, and normalized position. The second is features from a frozen, shallow encoder trained only once on the round-0 data. The third is the current student's decoder features.
Scale is manageable. Scribbles cover a few thousand voxels per slice-stack, and we subsample $P_i$, so each SVM is small. Only the band $U_i$ is evaluated, which sidesteps the quadratic-to-cubic cost of fitting SVMs on full volumes.
Bandwidth $\gamma$ and penalties are chosen per image by leave-strokes-out validation, holding out a fraction of each scribble. No ground-truth masks are needed. The complexity study then sweeps $\gamma$ and the backbone width jointly across scribble densities and cohort sizes. This maps where the best corrector capacity sits and whether it moves with label density as the second claim predicts.
Baselines
The naive reference point is a U-Net trained only on the scribbled voxels with a partial cross-entropy loss, with no pseudo-labels. Next to it we run plain self-training, in which the U-Net is retrained on its own thresholded predictions for several rounds. This reproduces the degradation pattern of the cyst paper and is the effect we claim to remove.
The first published-method baseline is the scribble pipeline of Learning to Segment Medical Images with Scribble-Supervision Alone: random-walk seeds, dense CRF, and Monte Carlo dropout filtering. It reports 88.0% Dice on ACDC (the Automated Cardiac Diagnosis Challenge cardiac MRI set) and 78.1% on NCI-ISBI 2013 prostate MRI. It is the closest competitor in spirit, because its uncertainty filter does the same job as our corrector.
The strongest existing method in the file is ScribbleVS. It reports 90.6% Dice on ACDC, above its own fully supervised baseline of 90.3%, and 88.4% Dice when trained on only 20 annotated cases. Since our claim is about small cohorts, the 88.4% figure at 20 cases is the target. Beating it means a statistically reliable Dice gain over ScribbleVS at 20 cases, and a larger margin at 5 and 10 cases, across repeated patient subsamples.
Reported numbers come from different scribble protocols and backbones; the two papers' fully supervised ACDC baselines are 91.2% and 90.3%. Our comparison is therefore internal. Every method gets:
- the same U-Net backbone and the same scribbles;
- the same patient splits, repeated over at least five random subsamples per cohort size;
- equal hyperparameter-search budgets measured in GPU-hours.
The SVM fitting time counts against our budget. For context on semi-supervised learning with dense labels on a few slices rather than scribbles, the multi-constraint self-training method of A Semi-Supervised Framework for Cardiac MRI Segmentation via Multi-Constraint Collaborative Self-Training reports 0.730 Dice at 5% annotations on ACDC. We cite it as context rather than as a matched baseline.
The ablations that isolate the key ingredient keep everything fixed except the corrector:
- Anchoring: the RBF-SVM trained on the previous round's pseudo-labels instead of on the strokes. This tests whether re-anchoring is what matters.
- Capacity endpoints: a linear SVM, standing for the low-$\gamma$ limit, and a k-nearest-neighbor vote, standing for the high-$\gamma$ limit.
- Graph-based alternative: label propagation on the same features.
- Trees: gradient-boosted trees on the same features.
- Feature source: the three choices of $\phi$ described above.
Why not just…?
Why not just keep the network's own uncertainty filter, as in the scribble-only pipeline? Monte Carlo dropout measures disagreement among versions of the same network. A systematic boundary error that all versions share passes the filter with high confidence. The cyst self-training plateau and decline, 75.87% to 74.94% (Self-Learning to Detect and Segment Cysts in Lung CT Images without Manual Annotation), is the kind of failure we expect from self-judgment. That evidence is indirect, since that pipeline used no dropout filter. The direct test is our per-round curve: we will plot Dice of the band labels against ground truth for dropout filtering and for the anchored corrector over five rounds. If dropout filtering already holds steady, our first claim fails.
Why not just use a simpler classifier, such as logistic regression or nearest neighbors, as the corrector? This is the most serious threat. The anchoring, not the kernel, may do the work. We do not argue around it. The RBF-SVM's bandwidth sweep includes both regimes as limits, and the ablations run linear and nearest-neighbor correctors directly. If either matches the tuned RBF-SVM across cohort sizes, we will report the result as "an anchored, capacity-matched corrector" and recommend the simpler model. The kernel then survives only as the tool for drawing the capacity curve.
There is prior evidence that intermediate capacity matters on small clinical data. Linear discriminant analysis reached 88% against 94.1% for an RBF-SVM on skin lesions in Characterization of digital medical images utilizing support vector machines. That study had 17 lesions, which is weak evidence. Our experiment is the real test.
Why not just use gradient-boosted trees, which are faster and need less tuning? On small clinical sets, tree ensembles were much less stable across folds. Pathological Voice Classification Using Mel-Cepstrum Vectors and Support Vector Machine reports a fold standard deviation of 0.0710 for XGBoost against 0.0160 for an RBF-SVM at equal mean score. FERMI at SemEval-2019 Task 5 reports a tree ensemble that topped the development set but fell on a shifted test set. Per-image correction on a few thousand stroke voxels is exactly that regime. Trees also give no slack-like quantity for flagging contestable pseudo-labels. Still, these results come from audio and text. The tree ablation, judged on both mean Dice and variance across subsamples, settles the question for segmentation.
Why not just refine pseudo-labels with a dense CRF? CRFs snap masks to intensity edges but trade away recall. Weakly Supervised Lesion Co-Segmentation on Ct Scans reports that its dense CRF "led to a sharp reduction in recall." The scribble-only pipeline found CRF hyperparameters delicate. A CRF also cannot use the expert strokes as hard evidence about which side of a weak edge a voxel belongs to. We include the CRF both alone and stacked with our corrector, so a complementary role would show up.
Why not just use a smaller or more regularized segmenter instead of adding a corrector? Shallower backbones do generalize better on small cohorts. Unveiling COVID-19 from Chest X-ray with deep learning found ResNet-18 beat ResNet-50. Temporal Convolution Networks for Real-Time Abdominal Fetal Aorta Analysis with Ultrasound found a shallow AlexNet-based model beat InceptionV4. But shrinking the segmenter costs boundary detail everywhere, while the corrector limits capacity only where labels are contested. The joint sweep over backbone width and $\gamma$ tests directly whether a narrow U-Net alone reaches the same Dice.
Proposed Experiments
The main benchmark is ACDC with scribble annotations. We will use the scribbles released with the cited scribble work if available. Otherwise we will generate them under a fixed, published-style protocol applied identically to all methods. Training cohorts are subsampled to 5, 10, 20, and all patients, and evaluation is on the standard held-out patients with full masks. NCI-ISBI 2013 prostate is the second benchmark, where the scribble-only pipeline loses 4.5% Dice relative to full supervision, leaving more room. Results on BraTS from the cited works are not comparable to a scribble setting, and the file reports no ScribbleVS BraTS number; we leave BraTS to a follow-up. If the lymphangioleiomyomatosis (LAM) lung-CT data is accessible, we will rerun its recursive self-training with our corrector. The anchors there would be the high-confidence regions of its unsupervised clustering teacher. We would test whether rounds three and beyond stop degrading.
Metrics are Dice and 95th-percentile Hausdorff distance per structure, plus two diagnostics:
- the accuracy of the band-voxel labels each round, measured against ground truth used only for analysis;
- the fraction of flagged pseudo-labels that were truly wrong, which is the precision of the slack-based filter.
Three checks decide the proposal:
- Degradation: plain self-training and dropout filtering lose Dice after round two in the low-cohort settings, while the anchored corrector does not. If both stay flat, the problem we target is not present on these benchmarks.
- Small-cohort gain: the full method beats ScribbleVS at 20 patients and by more at 5 and 10, under matched budgets.
- Capacity trend: the best $\gamma$ chosen by leave-strokes-out validation rises with scribble density, and the tuned RBF-SVM beats both its linear and nearest-neighbor limits. If the curve is flat, or an endpoint wins, the second claim fails and we report the simpler corrector.
References
- Self-learning to detect and segment cysts in lung CT images without manual annotation
- Neural network language models for low resource languages
- Learning to Segment Medical Images with Scribble-Supervision Alone
- ScribbleVS: Scribble-Supervised Medical Image Segmentation via Dynamic Competitive Pseudo Label Selection
- A Semi-Supervised Framework for Cardiac MRI Segmentation via Multi-Constraint Collaborative Self-Training
- Characterization of digital medical images utilizing support vector machines
- Pathological Voice Classification Using Mel-Cepstrum Vectors and Support Vector Machine
- FERMI at SemEval-2019 Task 5: Using Sentence embeddings to Identify Hate Speech Against Immigrants and Women in Twitter
- Weakly Supervised Lesion Co-Segmentation on Ct Scans
- Unveiling COVID-19 from CHEST X-Ray with Deep Learning: A Hurdles Race with Small Data
- Temporal Convolution Networks for Real-Time Abdominal Fetal Aorta Analysis with Ultrasound