When several doctors outline the same tumor or organ on a scan, they draw different boundaries. Models such as Pionono and the Latent Doctor Model now capture this by giving each doctor a small vector of numbers, a "style" code, that shifts the model's predicted outline toward that doctor's habits. The catch is the cost of the code. Learning a new doctor's style requires a pile of that doctor's full hand-drawn outlines, which is the expensive labor these models were meant to reduce. Wireless engineering has already faced the same problem. A base station that wants to steer its signal to a user needs to know that user's radio channel, and measuring the channel exactly uses up the bandwidth it is supposed to serve. The field's answer was that the transmitter rarely needs the full channel. One well-chosen bit, asked only of users for whom the answer matters, recovers most of the benefit.
This note proposes the same move for segmentation. Treat a clinician's style code as the unknown "channel." Instead of pixel outlines, estimate it from a handful of binary questions of the form "which of these two candidate contours is closer to how you would draw it?" Ask such a question only on cases where the plausible styles actually produce visibly different contours. The claim is that a low-dimensional style code (6–8 numbers in the models we build on) can be pinned down from a few dozen comparison clicks. The resulting personalized model should approach the agreement reached by a model given that doctor's full annotation history, at a small fraction of the clinician's time.
Core Claim
Let $z_r \in \mathbb{R}^d$ be the style code of rater $r$ in a trained multi-rater segmentation model with decoder $s(x, z)$, which maps an image $x$ and a code to a mask. A query on image $x$ shows two codes $a, b$, decoded as two masks. The rater answers with one bit:
$$y = \mathbb{1}\big[\mathrm{Dice}(m_r(x), s(x,a)) > \mathrm{Dice}(m_r(x), s(x,b))\big],$$
where $m_r(x)$ is the mask the rater would have drawn.
The system keeps a posterior $p(z_r \mid y_{1:t})$ over the rater's code. It chooses the next image and pair to maximize the expected information gain about $z_r$. It also has a gate: it asks a question on image $x$ only if the decoded masks under the current posterior disagree by more than a threshold $\tau$. Because each answer can at best halve the plausible region of code space, locating $z_r$ to precision $\epsilon$ in each of $d$ dimensions needs on the order of $d \log_2(1/\epsilon)$ bits. That count comes from the same volume argument behind codebook-based limited channel feedback, and for $d=8$ it predicts tens of questions rather than tens of full outlines.
What is new is not active learning for segmentation, and not rater modeling. It is the combination of three choices: the target of the queries is a persistent rater code rather than the current mask, the question format is a single comparison bit, and the queries are gated by predicted style-dependent disagreement.
Where This Sits
The rater models supply the object we estimate. Pionono gives each rater a Gaussian in an 8-dimensional latent space, with the mean as bias and the variance as within-rater inconsistency, learned offline from that rater's full labels. The Latent Doctor Model infers a doctor's code from a support set of their past annotations using a set encoder. Both assume the rater's labels already exist. We keep their decoders and replace the support set of masks with a budget of bits.
The closest interactive work is MedUHIP. It samples candidate segmentations and lets a clinician correct the current image with clicks and scribbles, adapting its sampling space to that clinician. MedUHIP's feedback is local to one image, uses fine-grained input, and is driven by the clinician with no explicit budget. Our queries are chosen by the system, are one bit each, and aim to fix a code reused on every later case.
The active-learning methods PAAL and ASSFT choose which scans to label in full. They treat the label as a single ground truth, so they cannot ask whose label or how coarse a label.
The wireless side supplies the design principles rather than code. Three results matter:
- One-bit side information and one-bit reward–punishment feedback recover most of the gain of full channel knowledge (Aggarwal and Sabharwal; Makki et al.). This motivates the comparison bit.
- Threshold-gated selective feedback beats full feedback once reporting overhead is counted (Karaca et al.). This motivates the disagreement gate.
- Channel memory makes a past measurement informative for future decisions (Exploiting Channel Memory for Joint Estimation and Scheduling). This corresponds to our assumption that a rater's style persists across cases.
We borrow these principles, not the analysis. The analogy fails in one respect: a rater's answer is a noisy human judgment, not a physical measurement. We model that noise explicitly using Pionono's per-rater variance.
Research Program
Base model. Train Pionono-style multi-rater models on public multi-rater datasets, holding out one rater at a time. The held-out rater's code is then unknown, and their masks serve only as the hidden truth that answers queries.
Query engine. Represent the posterior over $z_r$ with particles initialized from the population prior. Candidate pairs are drawn from posterior samples. Score each pair by expected information gain, estimated with Monte Carlo over particles. The gate computes, for each pending image, the spread of decoded masks under the posterior (one minus the mean pairwise Dice) and skips images below $\tau$. A noisy-response model, a logistic link on the Dice difference, keeps the posterior from collapsing on an answer that is inconsistent with the rater's earlier answers.
Simulated raters. The held-out rater answers each query by comparing the two candidates to their real mask. We add noise in two forms:
- Calibrated noise: the decision is flipped with probability tied to how close the two Dice values are.
- Within-rater noise: the reference is resampled from that rater's Pionono variance.
Small reader study. A small reader study then measures what a comparison bit actually costs in clinician seconds. The file gives contouring times, such as 200 seconds manually versus 1.3 seconds automatically for rectal cancer and 7.5 minutes for editing thoracic organs at risk (Deep learning for autosegmentation for radiotherapy). It gives no time per binary judgment, so this must be measured rather than assumed.
Baselines
Do nothing. The population prior is the multi-rater model with the held-out rater assigned the mean code, which is equivalent to predicting a consensus-like contour. If personalization from bits does not beat this, the idea fails.
Upper reference: full-label Pionono. This is Pionono with the held-out rater's code learned from all of their annotations. Pionono reports unweighted Cohen's kappa of 0.758 on Gleason2019 and 0.711 on the 25-rater breast cancer segmentation dataset (Pionono). Those are in-distribution numbers rather than leave-one-rater-out ones, so we rerun it under our protocol. The headline measurement is how much of the gap from prior to this ceiling we close per bit.
Strongest direct competitor: LDM support-set inference. The Latent Doctor Model infers a new doctor's code from $k$ of their full masks. On Gleason2019 its conditional variant reports KL divergence of 0.461 to the rater distribution. We vary $k$ from 1 to 20 on the same held-out raters and the same images, and plot accuracy against estimated clinician time. Our method beats it if its accuracy-versus-time curve lies above LDM's at matched clinician time, with time taken from the reader study.
Interactive competitor: MedUHIP-style clicks. MedUHIP is run with simulated corrective clicks on the same images. It reports 80% Dice in 2.72 interactions on REFUGE2, against 3.65 for SAM (MedUHIP). We compare at matched interaction counts, and we score accuracy on unseen later images from the same rater, where per-image correction does not carry over.
Ablations. Two ablations isolate the key ingredients:
- Random comparison pairs on random images, with no information-gain selection and no gate.
- Information-gain selection with the gate disabled.
All variants share the same trained decoder, the same image pools, and the same bit budget.
Why not just…?
Why not just ask the new doctor for a few full outlines and use the Latent Doctor Model? This is the real threat. If three outlines identify the code as well as forty bits, and three outlines cost less time than forty comparisons, the proposal loses. Masks carry far more information per query, but each also costs minutes. The matched-time comparison in the baselines settles this directly. We will report the crossover point honestly, including the case where full outlines win on small, simple structures.
Why not skip personalization and predict a consensus or an ensemble average? Consensus training discards how experts differ (Latent Doctor Model; Variational Inference for Quantifying Inter-observer Variability). The QUBIQ summary notes that simple ensembles mirror the panel average well, reporting top average Dice between 0.80 and 0.85. But the panel average is not any individual doctor. Personalization pays off only if differences between raters are large relative to image ambiguity. We first measure that variance split per dataset, and we expect the method to help only where the between-rater share is substantial.
Why not let clinicians click to correct, as in MedUHIP or SAM? Clicks fix the current image well. They are reactive, though, and the file gives no evidence that they transfer to the next case. Our test scores accuracy on images the rater never touched. A click-based method that also generalized its adaptation would weaken our claim, and that outcome would be visible in this experiment.
Why not use a standard active-learning score such as entropy or PAAL's predicted Dice to pick images? These scores measure model error against one ground truth and confuse genuine ambiguity with mistakes. PAAL itself shows entropy is overconfident, trailing PAAL by 4.9% on MSD Brain Tumour at a 5% budget (PAAL). We include PAAL-ranked image selection, using the PAAL-MedSeg code, as an alternative to our disagreement gate. This tests whether it matters that the gate targets style-dependent disagreement specifically.
Proposed Experiments
Data. We use multi-rater datasets with enough raters to hold out:
- Gleason2019 (6 pathologists)
- the 25-rater breast cancer segmentation set
- QUBIQ brain growth MRI (7 experts)
- REFUGE2 optic cup (7 ophthalmologists)
- LIDC-IDRI (4 radiologists), as a hard case with few raters
Metrics. Agreement with the held-out rater's own masks is measured as Dice or Cohen's kappa, depending on the dataset convention, and plotted against the number of bits and against estimated clinician seconds. QUBIQ's thresholded Q-Dice tracks whether the population-level distribution stays calibrated.
Decisive checks. There are three:
- On datasets with a large between-rater variance share, about 30 gated bits should close at least half the gap between the population prior and full-label Pionono. If after 50 bits the curve is flat, the idea is wrong.
- Removing the gate should waste bits on cases where the candidate contours look nearly identical, so the gated curve should rise faster.
- The bit-count scaling should follow roughly $d \log(1/\epsilon)$ when we vary the code dimension $d$ between 4 and 16. If many more bits are needed, rater noise dominates and the coarse-feedback analogy breaks down.
Secondary. Using sessions recorded at different times where available, we test whether a slowly updated posterior tracks a rater whose style drifts, in the spirit of feedback tracking on non-stationary channels (Alpcan).
References
- Probabilistic Modeling of Inter- and Intra-observer Variability in Medical Image Segmentation
- The Latent Doctor Model for Modeling Inter-Observer Variability
- MedUHIP: Towards Human-In-the-Loop Medical Segmentation
- Predictive Accuracy-Based Active Learning for Medical Image Segmentation
- Adapting Medical Vision Foundation Models for Volumetric Medical Image Segmentation via Active Learning and Selective Semi-supervised Fine-tuning
- Slotted Gaussian Multiple Access Channel: Stable Throughput Region and Role of Side Information
- Reinforcement-based data transmission in temporally-correlated fading channels: Partial CSIT scenario
- Joint Opportunistic Scheduling and Selective Channel Feedback
- Exploiting channel memory for joint estimation and scheduling in downlink networks
- Deep learning for autosegmentation for radiotherapy treatment planning: State-of-the-art and novel perspectives
- Variational inference for quantifying inter-observer variability in segmentation of anatomical structures
- QUBIQ: Uncertainty Quantification for Biomedical Image Segmentation Challenge
- Smart scheduling and feedback allocation over non-stationary wireless channels