Moving the Grade Boundaries: Sparse Target Labels for Cross-Site Ordinal Disease Grading

When a disease-grading model moves from one hospital to another, it often still detects disease but gets the stage wrong. Diabetic retinopathy is a typical case: there are five grades from none to proliferative, and a new camera can shift a "mild" eye into "moderate" without the model losing its sense of what disease looks like. We propose to treat this kind of failure as a boundary-relocation problem. A grading model can be read as a single severity score per image, plus four thresholds (cutpoints) that slice that score into grades. A handful of labeled cases from the new site, perhaps one to three per grade, are then used to re-place those thresholds. The much larger pool of unlabeled images from the new site is used to propagate soft, in-between grade labels. The representation itself is retrained only when a diagnostic test shows that the severity direction, and not just its thresholds, has changed.

The opportunity matters because fine-grained clinical models lose much of their value off-site. A neonatal pulmonary hypertension severity classifier fell from 0.84 to 0.63 F1 on held-out data (Ragnarsdóttir et al.). DeepTag fell from 0.824 to 0.634 weighted F1 out of domain (DeepTag). Meanwhile, the methods built to exploit a few target labels assume unordered classes. A cheap fix may exist, and nobody has checked whether it does.

Core Claim

Our claim has two parts, and both can be falsified. The first is empirical. On cross-site retinopathy and prostate grading, most of the accuracy lost between sites can be recovered by re-fitting the cutpoints along a frozen severity score using a few target labels. If that is true, the field's habit of reaching for heavy adaptation is mostly wasted effort for ordinal tasks. The second claim covers the remaining gap. When the severity direction itself shifts, unordered self-training mislabels the cases that sit between adjacent grades. A self-training stage that gives each unlabeled case a soft label, based on where it falls between labeled anchors of grade $k$ and $k+1$, should beat both unordered self-training and fine-tuning in that regime, and only in that regime.

The closest existing pieces each lack one ingredient. Selective Pseudo Labeling uses labeled target samples as distance anchors and raised pseudo-label precision from 35.2% to 61.6%. However, it scores a sample only against anchors of its predicted class, so a case halfway between two grades looks equally wrong for both. CLMT shows that sparse target labels get drowned out when trained jointly with source data, and it discards the source in a second, target-only stage. It was tested only on unordered DomainNet classes. ORUDA argues that clustering-based adaptation tears apart ordinal data that lies along a continuous path, but it uses no target labels. Hosseini et al. does use target labels on PANDA, but collapses Gleason grading to a binary task and names multi-grade staging as future work. Our proposal supplies the missing combination: ordinal geometry, sparse target anchors, and an explicit test of whether a cheap threshold fix is enough.

Research Program

The base model is any source-trained grader, for example a ResNet50 or a fundus foundation backbone, with an ordinal head. This head computes a scalar severity score $s(x) = w^\top f(x)$, where $f$ is the feature extractor and $w$ a learned direction, and predicts grade $k$ when $c_{k-1} < s(x) \le c_k$ for ordered cutpoints $c_1<\dots<c_{K-1}$.

The first component is anchor-plus-density cutpoint refitting. With $f$ and $w$ frozen, we re-estimate the cutpoints on the target site. The objective is the ordinal (cumulative-logit) likelihood of the few labeled target anchors. It is regularized by a term that prefers cutpoints in low-density regions of the unlabeled target scores, and by a prior that keeps the gaps between cutpoints close to their source values. Because only $K-1$ numbers are being fit, the drowning-out problem described by CLMT does not arise. All tuning uses leave-one-anchor-out likelihood on the labeled target cases, never a separate labeled validation set.

The second component is a shift diagnostic. We fit a target-only severity direction $w_t$ by ordinal regression on anchors plus the labeled target data, using strong shrinkage toward $w$. We then measure the angle between $w_t$ and $w$, along with how much anchor likelihood improves when $w$ is replaced by $w_t$. A small angle means the shift is a pure cutpoint shift, and we stop after the first component. A large angle triggers the third component.

The third component is between-anchor soft self-training. Following CLMT, the source is dropped and a student/teacher pair trains on target data only. The pseudo-labels differ from CLMT's, however. Each unlabeled case receives a soft Gaussian ordinal label, as in WSDA-OR. Its mean is set by interpolating the case's position between the nearest anchors of adjacent grades in feature space, and its width grows as the case nears the midpoint between anchors. In effect, the model is told "this case is about 2.4" instead of being forced to commit to grade 2 or grade 3.

To keep rare severe grades from being starved, a problem Hosseini et al. handled only in binary form, pseudo-label weights are balanced per grade.

We will also study label allocation. Given a fixed budget of target labels, we compare spending them uniformly across grades against spending them on cases the source model places near its cutpoints.

Baselines

The naive reference point is the source model applied to the target unchanged, with no adaptation. Every gain is reported relative to it, and this matches how GUES reports its results.

The second baseline is supervised fine-tuning. The source model is fine-tuned end to end on the same 1–3 labeled target cases per grade with an ordinal loss, using early stopping chosen by the same leave-one-anchor-out protocol. This is the default a practitioner would try.

The strongest existing semi-supervised method is CLMT (Two-Stage SSDA). It is the most competitive because it already solves the drowning-out of target labels and already exploits unlabeled target data with a teacher network. On DomainNet it reports a 10.2-point accuracy gain over MME at 1-shot and a 9.6-point gain at 3-shot. No reported result was found on Lacuna for CLMT on any retinopathy or prostate grading benchmark, so we will run it ourselves. It will use the same backbone, the same anchors, the same unlabeled pool, the same number of epochs, and an equal hyperparameter search budget under leave-one-anchor-out selection. Selective Pseudo Labeling is run under the same conditions as the strongest anchor-based competitor. We also run a zero-label reference: CauDR-style domain generalization (CauDR), which reports 61.82% average accuracy on 4DR against 55.42% for ResNet50.

Two ablations isolate the key ingredients. The first is cutpoint refitting from labeled anchors alone, without the density term. The second is our full third component with hard argmax pseudo-labels in place of the between-anchor soft labels.

Beating CLMT means a higher quadratic weighted kappa (agreement that penalizes far-off grades more) and a lower adjacent-grade confusion rate, averaged over seeds and anchor draws. The difference must be significant under a paired test across transfer pairs. It must also come at a lower cost, or be confined to the high-angle pairs as the second claim predicts.

Why not just…?

Why not just fine-tune the source model on the few target labels? With 5–15 labeled images, end-to-end fine-tuning moves millions of parameters to fit a handful of points. It also ignores the unlabeled pool, which is the weakness CLMT identifies in purely supervised adaptation. This is still a fair threat, and fine-tuning is run as a baseline under an identical tuning budget. If it matches cutpoint refitting, the conclusion is that "a few labels plus anything" suffices, and we will report that.

Why not just re-fit the thresholds and stop? This is the most serious alternative, and we have made it the first component rather than argue against it. If the diagnostic finds small direction angles on most pairs and the third component adds nothing there, the paper's main result becomes "threshold refitting is enough for cross-site grading." The soft self-training then survives only for the subset of pairs where the direction has rotated. We will state that outcome plainly.

Why not just run CLMT or Selective Pseudo Labeling with an ordinal head? These methods select or weight pseudo-labels by class identity, and ORUDA's argument implies that cluster-seeking objectives cut ordinal paths at the wrong places. The hard-label ablation of our own method tests this directly. If hard labels equal soft labels on high-angle pairs, the between-anchor mechanism is unnecessary.

Why not skip target labels and use domain generalization or test-time adaptation? CauDR still leaves 38% error on 4DR, and GUES reports a gain of only +2.8 QWK over the source model. Neither has any way to learn where the new site's grade boundaries lie. Including both as references measures how much the few labels are actually worth.

Proposed Experiments

The primary testbed is 4DR (IDRiD, DeepDRiD, Sustech-SYSU, DR2021), run as twelve single-source to single-target pairs. The second testbed is the APTOS 2019 / DDR / DeepDR / Messidor-2 suite with its twelve transfer pairs. For each, we vary the target labels per grade over 1, 3, and 5, using five random anchor draws. On PANDA Radboud→Karolinska we restore the full ISUP grades, which tests the multi-grade extension that Hosseini et al. left open; their binary result of 76.94% with 20% target labels is context only and not directly comparable. As an out-of-medicine check of the ordinal mechanism, we run AFAD→UTKFace with sparse target labels against ORUDA's unsupervised mean absolute error of 9.36. Metrics are quadratic weighted kappa, macro-F1, mean absolute grade error, recall on the two most severe grades, and expected calibration error.

The decisive check is a regression across all transfer pairs. It relates the measured direction angle to two quantities: the gain of cutpoint refitting over the source model, and the extra gain of soft self-training over cutpoint refitting. The first claim predicts that refitting recovers most of the source-to-oracle gap when the angle is small. The second predicts that the extra gain from soft self-training grows with the angle. If neither relation holds, the boundary-relocation view is wrong. A second check uses the label-allocation study: if labels placed near the cutpoints beat uniform per-grade labels at equal budget, that independently supports the claim that adjacent-grade boundaries are where target supervision matters most.