When several pathologists or dermatologists outline the same lesion, they rarely draw the same line. Most of that disagreement sits at a thin band of ambiguous tissue near the border, and each expert tends to resolve that band in a consistent way of their own. This proposal treats the image as a set of superpixels, which are small, contiguous patches of similar-looking pixels that follow local edges. It then treats each expert's mask as a series of include-or-exclude decisions about the superpixels in that band. Seen this way, an expert's "style" becomes a model with about ten numbers: how strictly they threshold a shared lesion score, and how much they lean toward including tissue with particular visible properties such as faint staining or weak edges. A model that small can be estimated from one to five annotated images of a new expert, a few-shot setting that pixel-level rater models handle poorly.
The opportunity matters because the settings where subjectivity is worst are also the settings with the least data. These are rare tumors, cohorts of under a hundred cases, and panels where any single rater annotated only a handful of images. Current multi-rater segmentation models learn rater behavior as an unconstrained latent vector over pixel features. They need many masks per rater and say nothing about why raters differ. We claim that moving the rater model onto superpixel decisions, with an explicit hierarchy from "global threshold only" up to "feature-dependent offsets", makes two things possible. Rater-specific predictions for unseen raters become accurate from very few examples, and the fitted coefficients become a readable, testable account of each rater's boundary preferences.
Core Claim
Let a shared network produce, for each superpixel $s$ of an image, a consensus lesion score $u_s$. Let $x_s$ be a short descriptor vector with roughly eight entries: mean stain or color intensity, texture energy, edge contrast on the superpixel's border, and signed distance to the consensus boundary. We model rater $r$ as including superpixel $s$ with probability
$$p_r(s) = \sigma\left(a_r u_s + b_r + w_r^\top x_s\right),$$
where $\sigma$ is the logistic function, $a_r$ is the rater's sharpness, $b_r$ is their overall inclusiveness, and $w_r$ is their sensitivity to specific visible properties. The style parameters $\theta_r = (a_r, b_r, w_r)$ get a Gaussian prior $\mathcal{N}(\mu, \Sigma)$ estimated from the training raters. A new rater's $\theta_r$ is the maximum-a-posteriori fit to their $k$ annotated images, with the shared network frozen. The claim has three parts. First, at $k \le 5$ this predicts a held-out rater's masks better than adapting a pixel-level latent model with the same $k$. Second, the advantage comes from the superpixel decision unit, not merely from having few parameters. Third, the feature terms $w_r$ capture real, stable structure beyond a global threshold. The third part is not guaranteed, and the experiments are built so that a negative answer is still informative.
Relation to Prior Work
Two lines of work model raters with latent variables, and they motivate the gap. Pionono gives each rater a learned 8-dimensional Gaussian whose mean encodes bias and whose variance encodes inconsistency. It concatenates samples with U-Net features and reaches unweighted κ of 0.758 on Gleason 2019 and 0.711 on a 25-rater breast cancer set. Its raters are indexed by ID, so a new rater requires fitting a new latent through a pixel-level decoder, and the latent has no anatomical meaning. The Latent Doctor Model instead embeds a support set of a rater's past decisions with a set encoder. This is the closest existing mechanism for few-shot rater transfer. It works on image-level grades, reporting KL 0.461 on Gleason2019 and entropy-discordance Spearman up to 0.728 on ISUP, not on masks. Like Pionono, it cannot say which tissue caused the disagreement. We keep their premise that disagreement is structured and rater-specific, but we replace the free latent with a small, interpretable decision model on superpixels.
Two further results tell us where disagreement lives and why it matters. Abhishek et al. show on IMA++ that agreement is lower for malignant lesions (mean Dice 0.753 against 0.791 for benign). Predicting a scalar agreement score from the image also helps diagnosis, with a 4.2% balanced-accuracy gain. Their prediction is one number per image; ours localizes disagreement to superpixels. MrPrism and DiFF produce calibrated consensus masks and rater confidence maps over pixels, but neither targets prediction for unseen raters.
Superpixels supply the method, and three papers also constrain it. Bian et al. use SLIC superpixels as instance tokens for Gleason grading. They find that randomly masking 50% of noisy instance labels improves macro AUC to 0.9429 on SICAPv2, so they treat subjectivity as uniform noise to be dropped. We treat it as structured signal to be modeled. SMMILe shows that SLIC "superpatches" plus smoothing recover full lesion extents where patch attention collapses onto obvious cores. That supports superpixels as the right unit for boundary-band reasoning. Cai et al. show that one-click "dominant labeling" of superpixels is very efficient, but they also warn that oversized superpixels straddle classes. SSENet reports high sensitivity to superpixel count. Together these mean a rater's mask may not be representable at a given granularity, so we measure that ceiling explicitly. SuperCL shows that superpixel-guided pretraining helps at 10–25% labels and that SLIC beats graph- and CNN-based superpixel methods. We use it only as an optional encoder initialization, and we use SLIC throughout.
Research Program
The work has four pieces. The first is a representability study. For each dataset and SLIC granularity, we compute the best Dice any superpixel-level labeling can achieve against each individual rater's mask. We compare this oracle against inter-rater Dice. If the oracle is not well above inter-rater agreement at a granularity the model can handle, the superpixel unit is losing exactly the signal we want to model, and we would move to a coarse-to-fine hierarchy of superpixels before going further.
The second is the shared model. A pretrained encoder, optionally SuperCL-initialized, feeds a light head trained on the pooled training raters. It outputs a per-superpixel consensus score $u_s$, trained against the fraction of training raters who include each superpixel. The descriptors $x_s$ are hand-defined, low-level, and computed without learning, so the coefficients $w_r$ stay readable.
The third is the rater model and prior. We fit $\theta_r$ for every training rater jointly with the head. We then estimate $\mu, \Sigma$ empirically and freeze everything except $\theta_r$ for new raters. We test a nested hierarchy: threshold only ($a_r, b_r$); then threshold plus feature offsets; then an optional pairwise term coupling adjacent superpixels on the region adjacency graph.
The fourth is the small-sample regime. Every experiment is repeated with the shared model trained on subsampled cohorts of 25, 50, and 100 images as well as the full set. This tests whether the structured rater model holds up when both images and per-rater annotations are scarce.
Baselines
The naive reference point is predicting the majority-vote consensus mask for every rater. This shows how much rater-specific modeling buys at all. The next reference is the strongest simple competitor: a global per-rater threshold on a pixel-level soft consensus map, with that one threshold fit on the $k$ support images. It shares our shared network's training data and encoder. If our full model does not beat it, the feature terms are not earning their keep.
The strongest existing method is Pionono. It reports unweighted κ of 0.758 on Gleason 2019 and 0.711 on the 25-rater breast cancer set, both in its standard seen-rater setting. We run it in two modes. In the seen-rater mode, we reproduce its protocol and check that our model is not worse than those numbers; this is the guard against having built a weaker model overall. In the unseen-rater mode, we freeze the trained network and fit a new rater's Gaussian mean and variance by gradient descent on the same $k$ support images. It gets the same encoder, the same cohort subsample, and the same hyperparameter search budget as ours, tuned on validation raters only. We also build a support-set variant in the style of the Latent Doctor Model: a set encoder over the $k$ support masks conditions the pixel decoder. This is the most natural "few-shot rater" design from prior work. It has no reported segmentation number on Lacuna.
For panel-level distribution quality, we add a deep ensemble, because QUBIQ found ensembles often beat variational models, with top-team Dice between 0.80 and 0.85 on its own tasks. Those are not our datasets, and we report no transferred number. Beating a baseline means a higher mean per-rater Dice or κ on held-out raters' held-out images at $k \in \{1, 3, 5\}$. The interval is a bootstrap over raters and images, and it must exclude zero at $k \le 3$.
Two ablations isolate the key ingredient. The first applies the identical logistic rater model with the same descriptor types to square patches and to individual pixels instead of superpixels. This separates the decision unit from the parameter count. The second removes the prior, separating hierarchical pooling from the representation.
Why not just…?
Why not just fine-tune Pionono's rater latent on the new rater's few images? This is the most direct competitor and is run as described above. Our reason to expect it to fall short at small $k$ is that its 8-dimensional latent acts through a pixel decoder. That decoder was never constrained to vary only near the boundary, so a handful of masks under-determines it. The k-shot curves settle the question. If Pionono's adaptation matches ours at $k=1$ and $k=3$, the claim fails.
Why not just learn one threshold per rater on a soft consensus map? This is a real threat. Much observed disagreement may be simple inclusiveness, and Abhishek et al.'s malignant–benign Dice gap is consistent with a threshold story. It is nested inside our model, so the comparison directly measures what the feature terms add. If they add nothing, we would report that rater styles in these datasets are essentially one-dimensional. That would be a useful and publishable negative result, and it would redirect the work toward the threshold model's few-shot properties.
Why not just average a pixel model's output within superpixels after the fact? If post-hoc superpixel pooling of Pionono's or the threshold baseline's output gives the same gain, the decision-unit claim is wrong. We run that pooled variant explicitly alongside the pixel- and patch-level ablations. Our model must beat it to support the claim that fitting rater decisions per superpixel, rather than smoothing pixels, is what helps.
Why not just use a deep ensemble? Ensembles model the panel, not a specific new rater, and training several networks on 25–50 images compounds overfitting. We still compare on panel-distribution metrics, where an ensemble may well win. A loss there would not undercut the rater-transfer claim, and we will report it plainly.
Proposed Experiments
We use three datasets: Gleason 2019 (244 cores, 6 pathologists), the 25-rater breast cancer segmentation set used by Pionono, and IMA++ (2,394 images, 5,111 masks, 15 annotators). The protocol is leave-raters-out. Training raters fit the shared model and prior. Validation raters tune hyperparameters. Test raters get $k$ support images and are scored on disjoint images. With only six raters, Gleason 2019 uses leave-one-rater-out. Metrics are per-rater Dice and unweighted κ against the held-out rater's mask, a boundary-band Dice restricted to pixels where training raters disagree, and Q-Dice over 10 thresholds for panel-level comparisons.
Five decisive checks follow. The first is the representability ceiling per granularity. Second, k-shot curves for our model, Pionono adaptation, the set-encoder variant, the threshold baseline, and post-hoc pooling, crossed with cohort sizes of 25, 50, 100, and full. We predict our advantage is largest at small $k$ and small cohorts and shrinks with full data. Third, the pixel-versus-patch-versus-superpixel ablation. Fourth, a stability test of interpretability. We refit $w_r$ on disjoint image halves for each rater with enough masks, mainly in the 25-rater set, and check that coefficient signs agree well above chance. Unstable coefficients would mean the "explanations" are noise. Fifth, a targeted reading. On IMA++, we test whether the fitted inclusiveness $b_r$ and edge-contrast weights shift systematically on malignant lesions, the case where disagreement is known to be higher.
References
- Probabilistic Modeling of Inter- and Intra-observer Variability in Medical Image Segmentation
- The Latent Doctor Model for Modeling Inter-Observer Variability
- What Can We Learn from Inter-Annotator Variability in Skin Lesion Segmentation?
- Multi-rater Prism: Learning self-calibrated medical image segmentation from multiple raters
- Calibrate the Inter-Observer Segmentation Uncertainty via Diagnosis-First Principle
- Multiple Instance Learning with Mixed Supervision in Gleason Grading
- SMMILe enables accurate spatial quantification in digital pathology using multiple-instance learning
- Revisiting Superpixels for Active Learning in Semantic Segmentation With Realistic Annotation Costs
- Symmetry-Aware Superpixel-Enhanced Few-Shot Semantic Segmentation
- SuperCL: Superpixel Guided Contrastive Learning for Medical Image Segmentation Pre-training
- QUBIQ: Uncertainty Quantification for Biomedical Image Segmentation Challenge