Medical imaging datasets are labeled by experts who often disagree, and a natural fix is to bring in an objective tiebreaker such as a blood biomarker. The trouble is that many biomarkers are not specific to cancer: CA19-9, PSA, CA-125 and alpha-fetoprotein (AFP) all rise with benign inflammation. Benign inflammation is also the situation that makes radiologists hesitate, because inflamed tissue can look like tumor. In those cases the readers and the blood test tend to make the same mistake on the same patients. Using the test to settle reader disagreement then confirms the error instead of correcting it.
This note proposes a way to clean labels that treats this shared failure as the central object of study. Each case gets a hidden true state with three values: benign, "benign mimic" (benign but inflamed or atypical, so both readers and non-specific markers are drawn toward a malignant call), and malignant. Every reader and every biomarker gets its own confusion pattern over these three states. Once the mimic state is named, a non-specific marker's agreement with a "malignant" vote no longer counts as independent evidence in exactly the cases where readers disagree. The claim is testable in a way a reviewer can check. On histology-verified cases, the fitted model should give AFP high mimic-to-malignant confusion and give a high-specificity cell-free DNA methylation assay low confusion. Downstream classifiers trained on the resulting labels should gain specificity at fixed sensitivity over every two-state alternative.
Core Claim
Existing label-noise methods use one of two error signals. The first is training dynamics: networks learn clean patterns before noisy ones. The second is agreement among humans. LiNC learns a trust weight per sample and clusters the weights into clean, ambiguous and noisy groups. Its authors concede that it fails when annotators share a systematic bias, because the network learns the shared error early. Variability Matters filters 120 pathologists by how well each agrees with one "anchor" annotator, who was chosen for annotation volume and not for correctness. Both papers motivate this proposal: each shows that human agreement is a poor proxy for truth when the bias is shared.
The tempting fix is to anchor labels on molecular evidence. The research file proposes this as Gap 1 and Gap 3. The biomarker-specificity literature argues against doing it naively. CA19-9 gives false positives across benign digestive inflammation (Arakawa et al.). Chronic hepatitis and cirrhosis disturb the same pathways as early hepatocellular carcinoma (Deng et al.). Exhausted PD-1+ T cells in lung lavage fluid cannot separate cancer from asthma or interstitial disease (Bal). These papers constrain the design. A biomarker cannot simply act as an extra annotator that is assumed independent of the readers given a binary disease state, because its false positives and the readers' false positives share a cause.
The model is written out as follows. Let $z_i \in \{\text{benign}, \text{mimic}, \text{malignant}\}$ be the hidden state of case $i$. Let $y_{ij}$ be reader $j$'s label, and let $b_{ik}$ be the discretized result of biomarker $k$. Then
$$ p(y_{i\cdot}, b_{i\cdot}, z_i \mid x_i) = \pi_\theta(z_i \mid x_i)\prod_j \Phi_j[z_i, y_{ij}] \prod_k \Psi_k[z_i, b_{ik}], $$
where $\pi_\theta$ is an image-conditioned prior over states computed by a neural network, $\Phi_j$ is reader $j$'s confusion matrix and $\Psi_k$ is biomarker $k$'s. Readers and markers are independent only given the three-state $z$. The dependence that a two-state model would ignore is absorbed by the mimic row: large entries of both $\Phi_j[\text{mimic},\text{malignant}]$ and $\Psi_k[\text{mimic},\text{high}]$ encode shared failure.
With several readers plus at least one biomarker there are three or more distinct views of each case. That is generically enough to identify a three-class mixture, which is the standard argument for multi-view latent-class models. Readers trained in the same tradition may still be correlated beyond $z$, so a small histology-verified subset is used to pin down the mimic row. The cleaned training target for a downstream classifier is the posterior $p(z_i = \text{malignant} \mid \cdot)$.
Research Program
The model above borrows its structure from two lines of work. The image-conditioned prior follows the Latent Doctor Model, which uses a conditional variational autoencoder to reproduce the distribution of opinions across a pathologist panel and to flag intrinsically ambiguous slides. That paper models only human disagreement and never asks which humans are biologically right; this proposal adds that missing anchor. The idea of judging annotations by diagnostic consequence comes from DiFF, which weights multi-rater masks by how much they help an image classifier. In DiFF the guiding signal comes from the same image, so image-level confounders reach both the masks and the diagnosis. Here the guiding signal is a non-image measurement, and its own non-specificity is modeled explicitly.
The program has three stages, all scored against histology. The first is semi-synthetic. The true labels are known, and the strength of shared failure between readers and biomarker is controlled directly. The second uses AMPLIFAI, 590 multiphase CT cases with dual independent reads, adjudication, and intermediate LI-RADS features such as arterial hyperenhancement, washout and capsule. Its authors document 10–30% variability in radiologist segmentation. This stage asks whether reader disagreement concentrates on lesions with atypical feature patterns, as the mimic hypothesis predicts.
The third stage requires new data collection, and no dataset in the file can substitute for it. AMPLIFAI does not include co-registered circulating biomarkers, and the breast radiomics–proteomics cohort of Li et al. has 180 subjects and a single operator's regions of interest. The planned cohort is cirrhotic patients with liver lesions. Each patient gets a multiphase CT read independently by at least three radiologists, serum AFP, and a cell-free DNA methylation assay in the style of NEEM-seq/DeepTrace (Deng et al.). That assay reports 98.5% specificity against benign liver disease. Histology or adequate imaging follow-up will be obtained on a subset. Pairing a non-specific marker (AFP) with a reportedly specific one (methylation) in the same patients is the design choice that makes the core claim falsifiable.
Baselines
The naive reference point trains a classifier on majority-vote reader labels and ignores biomarkers. Every method must beat it, and it shows how much each later step contributes.
The strongest existing label-side competitor is per-sample trust correction with LiNC. LiNC reports a mean last-epoch accuracy rising from 58.82% to 80.23% across ten MedMNIST tasks under 50% symmetric noise. It also reports a noise-detection AUC of 0.9837 on OrganSMNIST at 20% noise, against 0.8652 for AUM and 0.9126 for VoG. Those figures come from uniform synthetic noise. The structured, shared noise studied here is precisely LiNC's admitted failure case, so it is the right competitor to beat. LiNC will be run with its published three-phase schedule and the same backbone, epochs and hyperparameter search budget as the proposed method, on identical splits.
The Latent Doctor Model is the strongest disagreement-aware competitor. It reports a KL divergence of 0.461 on Gleason2019 and a Spearman correlation of 0.728 between its predictive entropy and panel entropy on ISUP. It will be trained on the reader labels only, with matched compute.
The strongest feature-side competitor is the dual-modality fusion of Li et al.. Their SVM reaches an AUC of 0.938 for benign versus malignant breast disease, against 0.841 for radiomics alone and 0.830 for proteomics alone. It will be reproduced here by feeding the biomarker as an input feature to the same image backbone and training on majority-vote labels.
The key ablation is the identical latent-class model collapsed to two states, which is a Dawid–Skene-style model with the biomarker as one more annotator. It isolates the mimic state as the only difference. A second ablation removes the histology subset to test whether multi-view identifiability alone is enough.
For DiFF-style diagnosis-first calibration, the file reports only relative AUC gains of up to 2.73% on glaucoma, 2.51% on thyroid cancer and 2.04% on melanoma. It reports no absolute numbers that could be reproduced. For AMPLIFAI as a diagnostic benchmark, the file contains no reported model result on Lacuna, so the baselines here will set the first numbers.
Beating the baselines means two things on held-out histology-verified cases. The proposed method must deliver higher specificity at matched 90% sensitivity than the best of the two-state ablation, LiNC, and input fusion. It must also shrink the gap in error rates between mimic and non-mimic benign cases, with bootstrap confidence intervals that exclude zero.
Why not just…?
Why not just feed the biomarker into the classifier as an extra input, as Li et al. did? This is the most serious threat. If input fusion trained on majority labels already reaches the specificity of the proposed method, then cleaning labels adds nothing. The counter-argument is that input fusion still learns from labels in which mimic cases were called malignant. The fused model may therefore learn "high AFP plus atypical appearance means malignant", which is exactly the shared error. The settling experiment is the comparison of input fusion with the proposed method at matched sensitivity, broken down by mimic versus non-mimic benign cases. If fusion is not worse on mimics, the label-side contribution is not needed and the program narrows to measuring shared failure.
Why not just use the biomarker as a tiebreaker only when readers disagree? Readers disagree most on mimic cases. Those are the cases where a non-specific marker is most likely to be falsely elevated, so tie-breaking selects precisely the regime where the marker is least informative. The semi-synthetic stage tests this directly: tiebreak accuracy should fall as the simulated reader–marker error correlation rises. If it does not fall, the mimic premise is wrong.
Why not just treat the biomarker as one more annotator in a standard two-class Dawid–Skene model? That model assumes the marker's errors are independent of the readers' errors given cancer status, which the specificity literature contradicts. It is run here as the key ablation. If its fitted confusion matrices already reproduce the predicted contrast between AFP and methylation, and it matches downstream specificity, then the third state is unnecessary, and the results will say so.
Why not just obtain histology for every case? Biopsy is invasive and not done for most lesions in this setting. LI-RADS exists to diagnose liver cancer without it, and Hao describes the harm of unnecessary prostate biopsies. The practical question is how small a verified subset is enough. The subset-size curve below answers it.
Decisive Experiments
The semi-synthetic test starts from MedMNIST v2 (Yang et al.), using BreastMNIST and PathMNIST. Benign images in a designated subclass or texture cluster become "mimics". Five simulated readers flip mimic labels to malignant at rate $r$. A simulated non-specific biomarker is elevated on mimics with probability $s$, and a simulated specific biomarker is not. The sweep runs over $r$, $s$, and the correlation between reader and biomarker errors on individual cases. Metrics are label-recovery accuracy, noise-detection AUC, and downstream AUC and specificity. The prediction is that the two-state model, tie-breaking and LiNC all degrade monotonically as the correlation increases, while the three-state model stays flat. If all methods degrade equally, the core idea fails.
On AMPLIFAI, the test fits the image-conditioned prior together with the reader confusion matrices, using resident and attending reads with adjudication. It then checks whether cases the model assigns to the mimic state are enriched for atypical LI-RADS feature combinations, such as hyperenhancement without washout or capsule. A 531/59 split is too small for strong claims, so this stage is used for hypothesis-checking, not headline numbers. The TCGA-LIHC subset has only 40 cases and is too small to support any molecular analysis.
The prospective liver cohort holds the decisive check. On histology-verified benign cases, does reader disagreement predict a false AFP elevation more than it predicts a false methylation result? The fitted mimic-to-malignant confusion should be clearly higher for AFP than for methylation. If the two are indistinguishable, shared failure is not real in this setting, and the simpler two-state model is the right tool.
The second readout is held-out specificity at 90% sensitivity against all baselines, stratified by mimic status. The third is a curve of specificity against the size of the histology-verified subset, which tells a clinical group how many biopsies the method actually needs.
References
- LiNC: Lightweight Noise Correction via Adaptive Label Refinement
- Variability Matters: Evaluating Inter-Rater Variability in Histopathology for Robust Cell Detection
- Determination and significance of a new carbohydrate antigen CA19-9 in digestive system cancers.
- Early detection of hepatocellular carcinoma via no end-repair enzymatic methylation sequencing of cell-free DNA and pre-trained neural network
- Evaluation of PD-1 T lymphocytes in bronchoalveolar lavage in lung cancer compared to benign lung diseases
- The Latent Doctor Model for Modeling Inter-Observer Variability
- Calibrate the Inter-Observer Segmentation Uncertainty via Diagnosis-First Principle
- AMPLIFAI: A Multiphase CT Dataset for Benchmarking Clinical Reasoning in LI-RADS Assessment of Liver Lesions
- A dual-modality machine learning precision diagnostic model integrated radiomics and proteomics for breast cancer
- Advanced nanomaterial for prostate cancer theranostics
- MedMNIST v2 - A large-scale lightweight benchmark for 2D and 3D biomedical image classification