Cancer registries store a pathologist's diagnosis, such as a prostate tumor's Gleason grade (a score of how abnormal the tissue pattern looks) or a breast tumor's hormone-receptor status, as a single fact. Researchers then feed that fact into survival models as if it were measured without error. Two lines of evidence undercut this. First, panels of expert pathologists often disagree on exactly these calls. Second, a vision model reading the same slide can catch outright misreads in hospital records. Language-model pipelines are now being built to pull more of these diagnoses out of free-text pathology reports and into registries. That will make the problem larger, because every extracted call gets recorded with the same false certainty.
This proposal builds registry fields that carry two things. The first is the call the reporting pathologist actually made, extracted from the report text. The second is an estimate, computed from the slide image, of how a panel of pathologists would have split on that same case. We then test whether survival analyses that use these two-part fields recover different, and more correct, estimates of how strongly grade predicts outcome than analyses that trust the recorded label. The central bet is that the report's wording is a poor guide to how contestable a diagnosis is. Pathologists write "Gleason 3+4" without hedging even on cases where half of a panel would say 4+3. The uncertainty therefore has to come from the tissue, not the text.
Core Claim
The claim has three parts. First, among cases that a panel of pathologists finds hard to grade, only a minority carry any hedging language in the report ("suspicious for," "cannot exclude"). A text-only uncertainty signal will therefore miss most of the contestable cases. Second, a slide model trained on multi-pathologist data can turn the reported call into a probability over what a panel majority would say. This probability tracks real panel disagreement on new cases.
Third, and most important, the statistics predict a specific effect of using this probability instead of the hard label. When an ordinal covariate like grade is randomly mislabeled, its estimated association with survival is pulled toward zero. Replacing the hard label with the probability should therefore strengthen the estimated grade–hazard relationship. The same correction should also widen confidence intervals honestly.
For readers new to the area: a hazard ratio says how much the risk of an event goes up per step of a covariate. Regression dilution is the name for this pull toward zero caused by mislabeled covariates.
Formally, let $x$ be the slide, $r$ the grade stated in the report, and $c$ the grade a panel majority would assign. A slide model trained on multi-rater data gives $p(y \mid x, z)$, the probability that a pathologist with latent reading style $z$ calls grade $y$. The field we store is
$$\pi(c \mid x, r) \propto P(c \mid x)\, P(r \mid c, x),$$
The first factor, $P(c \mid x)$, is the model's panel-majority distribution. The second, $P(r \mid c, x)$, is the probability that a random pathologist from the modeled population reports $r$ when the panel majority is $c$. Both are obtained by sampling $z$. The recorded call therefore enters as one observation of a noisy rater, not as ground truth.
Research Program
The work has three components.
The first component extracts grade and hedging from reports. TCGA pathology reports are scanned PDFs, so we first denoise the OCR text. For this we reuse the language-model summarization step from Multimodal Survival Modeling in the Age of Foundation Models, which cleaned the same reports. We then extract primary and secondary Gleason patterns, the grade group, and a hedging tag. The extraction runs in two phases. Phase one checks whether the field is present at all, following the two-phase parsing of mCODEGPT; that paper tested only on synthetic notes, whereas we run on real TCGA reports. Phase two adds the verbatim-evidence check from Self-Verification Improves Few-Shot Clinical Information Extraction: every extracted call must quote its supporting span, and the hedging tag is assigned from that quoted span. Both papers supply method here, and we reuse their techniques. Neither models diagnostic uncertainty, which is the gap this proposal fills.
The second component is a slide model that estimates panel disagreement. We train the conditional latent-doctor model of The Latent Doctor Model for Modeling Inter-Observer Variability on Gleason2019 (6 raters) and the ISUP biopsy cohort (20 raters). That model encodes each pathologist's reading style as a latent vector and was shown to reproduce panel distributions. We then apply it to TCGA-PRAD whole-slide images, using foundation-model tile features as input.
This transfer is the riskiest step. It crosses from tissue-microarray cores and biopsies to prostatectomy slides, and the latent-doctor paper never tried such a transfer. The latent-doctor work motivates this component, and our contribution relative to it is moving it from grading benchmarks into registry curation. TCGA reports do not identify individual pathologists, so we test submitting site as a coarse proxy for reading style when forming the support set.
The third component propagates uncertainty into survival analysis. We fit Cox models, the standard regression for time-to-event data, on the progression-free interval for TCGA-PRAD. We fit them in two ways. In the first, grade enters as the hard recorded label. In the second, grade enters through multiple imputation: we draw $c$ from $\pi(c \mid x, r)$, fit a Cox model on each draw, and pool the results with Rubin's rules. We run the same pipeline on TCGA-BRCA estrogen- and progesterone-receptor status. There we have no multi-rater labels, so we substitute an H&E receptor-prediction model like that of Clinical utility of receptor status prediction in breast cancer and misdiagnosis identification using deep learning on hematoxylin and eosin-stained slides. That work audited only structured labels, never report text, and confirmed misreads at a single center.
Baselines
The naive reference point is the recorded label used as-is. This means the TCGA structured grade field, or equivalently our extracted report call treated as certain, entered into a Cox model. This is how every registry study in the file operates, including the SEER analyses of treatment effects in nodular lymphocyte predominant Hodgkin lymphoma and metastatic colorectal cancer. To beat it, our approach must produce a grade hazard ratio that sits closer to the value obtained on a panel-regraded subset, and must do better on held-out calibration.
For extraction accuracy, the strongest reported number on narrative pathology reports is Clinical BioBERT at F1 0.959 on the primary-Gleason task in An Investigation into the Effects of Pre-training Data Distributions for Pathology Report Classification, measured on UCSF reports. We will fine-tune Clinical BioBERT on the same TCGA training split with the same tuning budget as our extractor. Matching it is sufficient here, because extraction is not the contribution. That paper also found that its model's representation outliers coincided with borderline reports. We will test whether those outliers predict panel disagreement as well as the slide model does.
For the disagreement estimate, the strongest competitor is the latent-doctor model itself. It reports a KL divergence of 0.461 on Gleason2019 and a Spearman correlation of up to 0.728 between its prediction entropy and 20-rater entropy on ISUP. Our target is to retain a Spearman correlation of at least 0.6 on TCGA prostatectomy slides against a new panel. That would show the transfer survives.
For survival prediction, we found no reported result on Lacuna for TCGA-PRAD specifically. The pan-cancer references are a C-index of about 0.795 for late fusion and 0.752 for summarized report text in Multimodal Survival Modeling in the Age of Foundation Models, and 0.6972 for Rasa on TCGA-BRCA. We will rerun the late-fusion embedding model on our exact splits. We do not expect to beat its C-index, and we are not claiming to.
The key ablation replaces the slide-derived distribution $\pi$ with three weaker alternatives. The first is a text-only distribution built from the hedging tag and model confidence. The second is a hard discordance flag, set when the slide call disagrees with the report. The third is a uniform-noise imputation at the same average error rate. This ablation isolates whether the slide-based estimate of each case's contestability is what does the work.
Why not just…?
Why not just read uncertainty from the report's hedging words, or ask the language model for a confidence score? This is the real threat, and it is cheap. If pathologists reliably hedged on hard cases, a hedging lexicon would do most of the job. Our premise that they do not is untested in the file. SUFO found borderline wording in only a subset of reports. The first decisive check below settles the question directly by comparing text-only uncertainty with slide-based uncertainty against panel entropy. If text matches slide within noise, the proposal collapses to a lexicon, and we would report that as the finding.
Why not just feed report and slide embeddings into a fusion survival model? For raw prediction this likely wins. Multimodal Survival Modeling in the Age of Foundation Models reaches a C-index of about 0.795 with black-box embeddings. But registries exist to estimate interpretable associations and to be audited. An embedding gives no grade field, no hazard ratio for grade, and no per-case flag a curator can act on. We will report fusion C-index alongside our results so the prediction cost of interpretability is visible.
Why not just flag slide–record discordance, as the receptor-status work did? A hard flag says only that the record looks wrong. It gives no graded probability that can be carried into an analysis. It also treats every concordant case as certain, even when a panel would have split. The flag-versus-distribution ablation measures whether the graded field changes hazard-ratio estimates beyond what the flags alone achieve.
Why not just use majority-vote labels from a re-read? Re-reading an entire registry is infeasible, which is the point of the exercise. The latent-doctor paper also shows that majority votes discard exactly the ambiguity we want to propagate. We use the majority only on the small regraded subset, as an evaluation anchor.
Proposed Experiments
We will recruit five pathologists to independently regrade 150 TCGA-PRAD slides without seeing the reports. The slides will be stratified by extracted grade group and by hedging tag.
The first decisive check estimates what fraction of high-disagreement cases carry any hedging language. It then compares text-only and slide-based uncertainty on predicting panel entropy, using Spearman correlation with bootstrap confidence intervals. If slide-based uncertainty does not clearly beat text-only uncertainty, the core claim fails.
The second check runs a simulation on the full cohort. We inject known misclassification into grade and confirm two things: the hard-label Cox model attenuates the hazard ratio as dilution theory predicts, and multiple imputation under $\pi$ recovers the true value.
The third check is the real-data test. On TCGA-PRAD, and on TCGA-BRCA with receptor status, we compare hazard ratios under hard labels, under $\pi$-based imputation, and under the ablations. We also compare against the panel-majority estimate on the regraded subset. The prediction is falsifiable: the $\pi$-based hazard ratio should lie farther from the null than the hard-label estimate and closer to the panel estimate.
Finally, we will send the 50 cases with the highest $\pi$-versus-record discordance for blinded review. This measures how often the field surfaces genuine misreads. It follows the audit logic of the receptor-status work, extended to cases whose report text supplied the recorded call.
References
- Multimodal Survival Modeling in the Age of Foundation Models
- Introducing mCODEGPT as a zero-shot information extraction from clinical free text data tool for cancer research
- Self-verification improves few-shot clinical information extraction
- The Latent Doctor Model for Modeling Inter-Observer Variability
- Clinical utility of receptor status prediction in breast cancer and misdiagnosis identification using deep learning on hematoxylin and eosin-stained slides
- The stage‐specific roles of radiotherapy and chemotherapy in nodular lymphocyte predominant Hodgkin lymphoma patients: a propensity score‐matched analysis of the SEER database
- Lymph node status as a prognostic factor after palliative resection of primary tumor for patients with metastatic colorectal cancer
- An Investigation into the Effects of Pre-training Data Distributions for Pathology Report Classification
- Enhancing WSI-Based Survival Analysis with Report-Auxiliary Self-Distillation