When a pathology model trained at one hospital is moved to another, it gets worse. Clinicians are often willing to help it recover, but their time is scarce, and on many tissue features they disagree with each other anyway. Concept bottleneck models predict a diagnosis through a layer of human-readable features, such as "cribriform glands" or "poorly formed glands." That layer gives a pathologist a natural place to step in: they can correct a feature rather than overrule the whole diagnosis. Existing interactive versions of these models choose which feature to ask about by how uncertain the model is, and they treat the human's answer as ground truth. This combination is wasteful in pathology. Many of the features where the model is most uncertain are ones that pathologists themselves split on. On those, a single expert's answer is close to a coin flip from a panel, so asking spends a query and buys almost nothing.
We propose to split the model's uncertainty about each feature into two parts. The first is the part a panel of pathologists would also have, which we call rater ambiguity. The second is the excess, which reflects the model's own unfamiliarity with the new site's images. We then ask the local pathologist only about features with high excess uncertainty, and fold each answer back in as one noisy vote whose weight depends on how reliable a single rater is for that particular feature. The bet is that, on an external cohort and for a fixed number of pathologist questions, this recovers more grading accuracy and better matches the panel's spread of opinion than either uncertainty-driven querying or fine-tuning on a few relabeled cases.
Core Claim
The claim rests on one premise and produces one consequence. The premise is that disagreement among raters about a feature belongs mostly to the tissue morphology, so it should carry over across sites. The model's excess uncertainty belongs to the acquisition domain (stain, scanner, preparation), so it should grow under site shift.
Write $H_k$ for the model's total predictive entropy on concept $k$. The standard decomposition splits it as $H_k = A_k + E_k$:
- $A_k$ is the expected entropy across an ensemble trained on soft multi-rater labels. Because each member is trained to reproduce the panel's spread, $A_k$ estimates rater ambiguity.
- $E_k$ is the disagreement among ensemble members (mutual information), which estimates the model's unfamiliarity.
Querying by $E_k$, weighted by the concept's influence on the grade and the cost of asking, should dominate querying by $H_k$ once the model leaves its training site. The decomposition itself is standard. What is new is using it to decide when to interrupt a fallible human expert after deployment shift, and pairing it with an answer model that treats the expert as one draw from a panel.
The closest work is CooP in Interactive Concept Bottleneck Models. CooP ranks concept queries by uncertainty, importance and cost, and it assumes a perfect oracle. It also reports results only on CUB, CheXpert and knee X-rays, with no site shift. We keep its importance and cost terms but change both the uncertainty term and how answers are used.
Pathologist-like explainable AI for interpretable Gleason grading in prostate cancer (GleasonXAI) supplies the method we build on: a bottleneck over 44 sub-patterns annotated by 54 pathologists and trained on soft labels. It reports Dice $0.713 \pm 0.003$ versus $0.691 \pm 0.010$ for direct pattern segmentation. GleasonXAI was evaluated only on internal tissue microarray (TMA) cores, with no interaction and no external cohort.
The Latent Doctor Model motivates the work from the other side. It models the full panel distribution and flags high-entropy cases for review, but its deferral works at the whole-case level, has no concept layer, and is not tested externally.
Research Program
Base model. We train a concept bottleneck ensemble on the GleasonXAI cores. Each concept head is trained with the soft-label loss used in that paper, and a grade head sits on top of the concepts. Because the targets are panel distributions, each member learns rater ambiguity directly, and the ensemble spread supplies the excess term. We will also test a single-model variant that uses the Gaussian concept embeddings of Probabilistic Concept Bottleneck Models to estimate the excess term instead of an ensemble. Because that paper warns that unconstrained side channels can leak information around the bottleneck, we use no residual channel at all.
Answer model. When a local pathologist answers a query, we do not overwrite the concept with a hard 0 or 1. We instead set it to the probability that the panel majority holds the feature, given that one rater said yes (or no). This probability is estimated separately for each of the 44 concepts from the GleasonXAI multi-rater annotations, by holding out one rater at a time.
This directly addresses a limitation of The CrowdGleason dataset. There, each annotator's reliability is a single static confusion matrix that ignores morphology. Here, reliability is specific to each feature.
Site-shift diagnostic. The same machinery gives a concept-level breakdown of external failure, which addresses the open question of whether transport breaks the image-to-concept step, the concept-to-grade step, or the raters' conventions. All models are local argues that locked models degrade on transport, but treats the model end to end. We can ask where inside the model the degradation happens. Three comparisons answer this:
- If the external drop is mostly carried by rising $E_k$, the problem is in the image-to-concept step, and concept queries should fix it.
- If the grade head is wrong even when it is given the local pathologists' concept annotations, the concept-to-grade relationship has shifted, and concept queries will not help.
- If the local pathologists' disagreement profile differs from the training panel's, rater conventions have shifted.
Data collection. External concept labels do not exist, so we will collect them. Three pathologists will annotate the 44 sub-patterns on a subset of roughly 100 cores from Gleason2019. Gleason2019 already carries grade labels from six pathologists. We will first check the provenance of both datasets to rule out institutional overlap with GleasonXAI. The new annotations serve as real query answers and as the test of whether rater ambiguity transports.
Baselines
Locked model. The naive reference point is the GleasonXAI-style bottleneck deployed unchanged on the external cohort with no queries. Every other method must improve on it per unit of pathologist effort.
CooP. The strongest existing interactive method is CooP, run with its published scoring rule: total concept uncertainty times importance divided by cost. Its answers are treated as hard ground truth. CooP reports 5–10% higher accuracy after 5 interactions on CUB and CheXpert. There is no reported result on Lacuna for CooP on any histopathology or Gleason benchmark. For fairness, CooP will use the same trained ensemble, the same concept-importance estimates, and the same cost model, so the comparison changes only the uncertainty term and the answer model.
Isolating each ingredient. Two ablations separate the contributions. The first uses excess-uncertainty querying with hard answers. The second uses total-uncertainty querying with soft answers. Together they show whether the gain comes from choosing better questions or from interpreting answers better.
Whole-case deferral. This follows the Latent Doctor Model: entropy-based deferral of entire cases, scored as the pathologist's grade replacing the model's. We will compare at matched pathologist minutes, with per-case and per-concept times measured from our annotators.
Local fine-tuning. This follows the recurrent local validation idea: label-level fine-tuning of the grade and concept heads on a small set of locally graded cases, given the same total annotation time.
Reference numbers from other aggregation methods. The best numbers in the file for these settings come from distribution-aware aggregation. CrowdGleason's SVGPMIX reaches Cohen's quadratic kappa 0.7814 on SICAPv2. The conditional Latent Doctor Model reaches KL divergence 0.461 to the panel distribution on Gleason2019.
What counts as beating them. We beat a baseline if the area under the curve of quadratic kappa against the number of pathologist questions is higher, with a 95% bootstrap interval excluding zero. The second criterion is a reduction in KL divergence to the six-rater Gleason2019 distribution. On Gleason2019, the locked model should land in the neighbourhood of the Latent Doctor Model's 0.461 and then improve on it with queries. The first criterion is the primary endpoint.
Why not just…?
Why not just fine-tune on a few locally labeled cases? This is the most serious threat. All models are local argues that local adaptation is exactly what deployed models need, and grade labels are faster to give than 44 concept judgments. Concept queries can only win if the external failure sits in the image-to-concept step and the questions are targeted.
The matched-minutes comparison and the shift diagnostic will settle this. If the diagnostic attributes most of the drop to a shifted concept-to-grade relationship, fine-tuning should win, and we will report that as the finding rather than defend the query method. A hybrid that fine-tunes the concept heads using query answers is a natural follow-up if both help.
Why not just use CooP with total uncertainty? Under site shift, total and excess uncertainty might rank concepts almost identically, and then our method would add nothing. GleasonXAI's soft labels show substantial disagreement on borderline sub-patterns, and the Latent Doctor Model reports a Spearman correlation up to 0.728 between model and human entropy on ISUP. Both suggest that much of the model's uncertainty is rater-shaped, which predicts that total-uncertainty querying will waste queries. We will measure the rank correlation between $H_k$ and $E_k$ on external cores directly. If it is high, the ablation will show no gap and the claim fails.
Why not just remove the shift with stain normalization or a large pathology foundation encoder? If better features close the external gap, there is little left for humans to fix. The file gives mixed evidence. Navigating the MIL Trade-Off finds that a small domain-specific encoder with Maxsoft pooling outperformed Prov-GigaPath on several metrics at far lower cost, which does not suggest that scale alone solves transport. We will still run every method on both a domain-specific encoder and a foundation encoder, with and without stain normalization. We will report how much excess uncertainty survives each, because the method's value is proportional to that residue.
Why not just defer the whole case? Deferral is simple and needs no concept layer. However, it gives the pathologist no indication of which feature the model is unsure about, and it spends a full case read where one targeted question might do. The matched-minutes comparison with Latent Doctor-style deferral decides this.
Proposed Experiments
Training uses the GleasonXAI cores, with cross-validation over cores for internal numbers. External evaluation uses Gleason2019 (TMA cores, six graders) as the primary cohort, because its multiple graders allow distributional metrics. SICAPv2 patches, which have pixel-level grade labels from a different institution, serve as a second external cohort for grade-only metrics. All work stays at core or patch level, which avoids gigapixel aggregation as a confound.
Query answers come from two sources. On the annotated Gleason2019 subset we use our three new annotators. On everything else we simulate answers by drawing a held-out GleasonXAI rater's annotation, which is pessimistic because those raters see out-of-site images only in the simulation sense.
First check: does rater ambiguity transport? For each concept, we compare the disagreement entropy of the training panel with that of our three external annotators, using rank correlation and calibration of $A_k$ against the external annotators. If the model's ambiguity estimate is badly miscalibrated externally, the decomposition is unreliable, and the project narrows to the shift diagnostic alone.
Second check: does excess-uncertainty querying recover faster? We plot recovery curves for 0 to 10 queries per core and compare our method, CooP and the two ablations. Quadratic kappa against the six-grader consensus is the primary endpoint, and KL divergence to the six-grader distribution is the secondary one.
Third check: does it hold up in pathologist time? We repeat the comparison on the pathologist-minutes axis against local fine-tuning and whole-case deferral, using measured annotation times.
The first check decides whether the premise holds and the second whether the query rule matters. The third decides whether concept-level interaction is worth a clinician's time at all.
References
- Interactive Concept Bottleneck Models
- Pathologist-like explainable AI for interpretable Gleason grading in prostate cancer
- The Latent Doctor Model for Modeling Inter-Observer Variability
- Probabilistic Concept Bottleneck Models
- The CrowdGleason dataset: Learning the Gleason grade from crowds and experts
- All models are local: time to replace external validation with recurrent local validation
- Navigating the MIL Trade-Off: Flexible Pooling for Whole Slide Image Classification