Medical segmentation models are expensive to train because every pixel-level mask has to be traced by a specialist. Most work on reducing this cost asks the model which images it is unsure about, and then requests full masks for those images. This proposal starts from a different observation. Once a model is deployed on unlabeled scans, the quantity we care about, how wrong it is on each image, is hidden. The model's own confidence is a poor stand-in for it. Cheap expert signals such as a scribble, the two RECIST tumor-diameter lines a radiologist already draws, or eye-tracking fixations recorded during reading are not only weaker training labels. They are also noisy measurements of that hidden error. We propose to treat labeling as a problem of acting under partial observability, meaning the agent cannot see the true state and must keep a probability distribution, or belief, over it. In this formulation the hidden state is the per-image error of the current model, the actions are queries of different cost and informativeness, and each cheap annotation updates the belief before the agent decides whether a full mask is worth paying for.
The payoff is a labeling policy that spends expert time where the model is actually failing, and not only where it reports being unsure. Because the same cheap annotations that sharpen the belief also serve as training signal, they pay off twice. In two sentences: current active learning estimates where the model is wrong from the model's own outputs, which fails exactly when the model is confidently wrong. We instead let cheap expert marks act as observations that correct a belief about the model's hidden errors, and we choose the next query (which image, and which kind of annotation) by its expected error reduction per minute of expert time.
Core Claim
Our central claim is narrow and testable. A belief over the per-image Dice score of the current model predicts true unlabeled-image accuracy better than a predictor that sees only the model's outputs, provided the belief is conditioned on cheap weak annotations such as scribbles, RECIST lines, and gaze. Feeding this belief into a cost-aware acquisition rule should reach a target Dice at lower total annotation cost than accuracy-prediction active learning or uniform weak supervision. Our secondary claim is that looking a few queries ahead helps beyond a one-step greedy choice. This holds only because the hidden error state drifts each time the model is retrained. We treat this second claim as an open question with a planned test, not as an assumption.
Positioning Against Prior Work
The closest existing work falls into two camps, and neither combines the two ingredients we combine.
The first camp turns labeling into sequential decision-making. Reinforced active learning for image segmentation trains a deep Q-network to choose which image regions to send to a human oracle, with change in validation mIoU as the reward. It treats the problem as fully observable, assumes a noise-free dense oracle, and must retrain the segmenter just to compute each reward. We drop all three assumptions. Our state is hidden, our oracle offers several annotation types of different fidelity, and our reward signal comes from the belief rather than repeated retraining. MedVR frames medical image reading as acting under hidden high-resolution evidence, using a zoom tool trained with consensus rewards. However, it never queries a human, outputs answers and boxes rather than masks, and keeps no explicit belief. Our action space is the set of human queries that MedVR's discussion of its limitations leaves open.
The second camp makes active learning label-efficient without any notion of hidden state. Predictive Accuracy-Based Active Learning (PAAL) trains an auxiliary network to forecast the Dice of unlabeled images from the segmenter's probability maps. It is our strongest competitor and the clearest contrast. PAAL's accuracy estimate is a function of the model's own outputs alone, so it cannot be corrected by external evidence. Our belief is the same kind of estimate, but it gets updated by expert observations. The multi-fidelity methods D-MFDAL and BMFAL-BC supply the idea of choosing both a point and a fidelity level under a cost budget. They were developed on PDE simulators where fidelity means mesh resolution and where low fidelity is a deterministic coarsening. We borrow their cost-normalized information gain but apply it to stochastic human annotators, whose low-fidelity output is a different kind of object (a line or a fixation map, not a blurred mask).
Several papers supply method pieces. Optimality Guarantees for Particle Belief Approximation of POMDPs justifies representing the belief with a modest set of particles. Offline Meta Learning of Exploration shows how to learn belief inference from logged data and warns that logged data may never contain the transitions needed to tell latent states apart. That warning matters here, because a warm-start pool labeled only with full masks says nothing about how scribbles relate to error. ASTC shows that belief-space search with value bounds works on clinical logs, but only on tabular vital signs. Inference and Learning for Active Sensing explicitly concedes that its expected-utility sampling does not cover hidden-state settings, and that concession is the gap we occupy.
The weak-supervision papers supply the observation channels and also constrain the design. Scribble supervision shows that scribbles alone come within a few Dice points of full masks. RECIST co-segmentation turns routine diameter lines into masks. GazeRefine turns passive gaze into training-free segmentation. Two of these papers also report failure modes that motivate us. Pseudo-label loops amplify early mistakes, and GazeRefine is weak on low-contrast prostate MRI. A belief over per-image error is meant to flag exactly those images for escalation to a full mask.
Research Program
The system has four parts.
The first part is a segmenter: an nnU-Net configuration following The Medical Segmentation Decathlon. It is trained on whatever mixture of full masks and weak annotations has been acquired so far, using the scribble loss and random-walk seed expansion of the scribble paper for weakly labeled images.
The second part is a belief model over each unlabeled image's error. The model starts from the segmenter's probability maps, as PAAL's predictor does. It then takes as optional inputs any weak annotations already collected for that image and outputs a distribution over that image's Dice. A scribble that crosses the predicted boundary, or a RECIST line whose length disagrees with the predicted lesion extent, should shift this distribution. We represent it with a small particle set, or equivalently a discretized distribution. This lets us carry the belief forward across retraining rounds with a learned drift step: the belief about image $i$ after retraining depends on how much the model changed on similar images.
The third part is an observation model for each annotation type $k$, estimated on a small pool where both full masks and weak annotations exist. For scribbles and RECIST lines we can simulate the weak annotations from masks, as prior work does. For gaze we can use the real fixations paired with Kvasir-SEG.
The fourth part is an acquisition rule. At each step the agent picks the image $i$ and annotation type $k$ (scribble, RECIST-style lines, full mask, or gaze where available) that maximize
$$ \text{score}(i,k) = \frac{\mathbb{E}_{b_t}\left[\Delta \mathcal{E}(i,k)\right]}{c_k}, $$
where $b_t$ is the current belief, $\Delta \mathcal{E}(i,k)$ is the predicted reduction in summed hidden error over the unlabeled pool after the query and its effect on retraining, and $c_k$ is the expert time cost of annotation type $k$. We compare a one-step version of this rule against a short-horizon belief-tree lookahead of depth two to three, with pruning by value bounds in the style of ASTC. The lookahead can capture strategies such as "scribble now, and escalate to a full mask only if the scribble reveals a large error."
Costs are a real uncertainty. The file gives no medical per-annotation timings, so we will treat the ratios between scribble, line, and mask cost as parameters and report results across a sweep, not commit to a single invented value.
Baselines
The naive reference point is random image selection with full masks under the same cost budget. A second naive point is uniform weak supervision, which spends the entire budget on scribbles for as many images as it affords. This is a serious baseline. The scribble paper reports 88.0% Dice against 91.2% for full supervision on ACDC, and 78.1% against 81.8% on NCI-ISBI prostate. Any method that wastes budget on full masks could lose to it.
The strongest existing method is PAAL. It reports annotation-cost reductions of 50% to 80% across its benchmarks. On the MSD brain tumour task at a 5% budget it reports beating maximum-entropy selection by 4.9% and loss-prediction active learning by 20.7%. No absolute Dice value for PAAL on ACDC or SegTHOR was found on Lacuna. We will run PAAL from its published description with the same segmenter, warm-start pool, and per-round query batch size. We will also give it a cost-aware extension: PAAL's predicted Dice divided by annotation cost, choosing among annotation types. This keeps the comparison from reducing to "we allowed cheap queries and they did not."
We include the reinforced active learning DQN as the fully observable sequential baseline, with its region actions replaced by whole-image queries. Its retraining cost will be reported separately. For reference, it reports needing about 30% less labeled data than BALD to reach target accuracy on Cityscapes. We also include a BMFAL-BC-style greedy batch information-gain baseline over (image, fidelity) pairs.
All methods get the same total expert-cost budget, the same segmenter compute per round, and the same hyperparameter tuning budget on a held-out validation split. Beating PAAL means a lower cost-to-target-Dice on at least three of the four datasets, at a cost-ratio setting fixed before the test runs, with non-overlapping intervals across five seeds.
Two ablations isolate the key ingredient. The first removes weak-annotation observations from the belief model. This leaves a PAAL-like predictor inside our acquisition rule, and it tests whether observations, not the planner, carry the gain. The second replaces lookahead with the one-step rule, to test whether the sequential, hidden-state framing adds anything beyond greedy value of information.
Why not just…?
Why not just use PAAL with costs attached? This is the most likely competitor, and our first ablation is effectively this method. Our argument is that PAAL's predictor cannot detect a confidently wrong image, because it sees only the confident output. The decisive measurement is the rank correlation between predicted and true Dice on unlabeled images, computed separately for the images where the segmenter is most confident. If adding scribble observations does not improve that correlation, our main claim fails and we will say so.
Why not just scribble everything? Scribbles are cheap and close to full supervision on ACDC. The remaining gap, though, is concentrated on hard images, and GazeRefine's weaker prostate result suggests weak signals fail precisely on low-contrast targets. The uniform-scribble baseline settles this directly. If our method cannot beat it at the target Dice of full supervision, the escalation logic is not worth its complexity.
Why not just a greedy one-step rule, or a contextual bandit, without any lookahead? This is a real threat. Reinforcement-learned acquisition has a mixed track record against greedy heuristics, and our hidden state may drift too little between rounds for planning to pay off. The second ablation tests this. If depth-one matches depth-three, we will present the contribution as belief tracking with observations, not as planning, and drop the heavier machinery.
Proposed Experiments
We will use four datasets. ACDC and NCI-ISBI prostate MRI cover scribble and full-mask actions with simulated scribbles. MSD brain tumour and SegTHOR give comparison with PAAL. Kvasir-SEG provides real gaze fixations; GazeRefine reports 88.10% Dice training-free there, against 88.41% for nnU-Net. DeepLesion provides real RECIST lines, where the co-segmentation paper reports 90.3% mean Dice on its 1,000 manually segmented lesions.
The metrics are Dice as a function of cumulative expert cost, cost needed to reach 95% and 99% of full-supervision Dice, and calibration and rank correlation of the per-image error belief against true Dice.
The first decisive check is the belief-quality test from the PAAL question, run before any acquisition experiment. If it fails, the acquisition experiments are moot. The second decisive check is the cost sweep. The method should win across a wide range of scribble-to-mask cost ratios, not only at a ratio favorable to it. The third check targets the offline-ambiguity issue raised by BOReL. We will train the observation model on a pool labeled only with full masks plus simulated weak marks, then test on real RECIST lines and real gaze. A large drop would mean the observation model must be learned online, and we would report that cost.
References
- Reinforced active learning for image segmentation
- MedVR: Annotation-Free Medical Visual Reasoning via Agentic Reinforcement Learning
- Predictive Accuracy-Based Active Learning for Medical Image Segmentation
- Disentangled Multi-Fidelity Deep Bayesian Active Learning
- Batch Multi-Fidelity Active Learning with Budget Constraints
- Optimality Guarantees for Particle Belief Approximation of POMDPs
- Offline Meta Learning of Exploration
- The Actor Search Tree Critic (ASTC) for Off-Policy POMDP Learning in Medical Decision Making.
- Inference and Learning for Active Sensing, Experimental Design and Control
- Learning to Segment Medical Images with Scribble-Supervision Alone
- Weakly Supervised Lesion Co-Segmentation on Ct Scans
- GazeRefine: Expert Gaze as a Test-Time Prompt for Training-Free Medical Image Segmentation
- The Medical Segmentation Decathlon