Primary progressive aphasia (PPA) is a family of neurodegenerative diseases whose first symptom is a breakdown of language. Clinicians sort patients into three variants: non-fluent, semantic and logopenic. The sorting matters because each variant points toward different underlying brain pathology. Yet 20 to 41 percent of patients do not fit any variant cleanly (Montembeault et al.). For these patients, different experts plausibly reach different diagnoses. That disagreement has never been measured. Published AI diagnosers are scored only on clean, prototypical cases. This proposal does three things. It collects a multi-clinician diagnosis panel for PPA that deliberately includes the mixed cases. It gives the clinicians patients' scores on non-verbal tests, meaning tests that do not require understanding instructions or speaking. It then asks whether those language-free scores can predict how the panel splits.
The underlying bet is that clinician disagreement in PPA is not random noise. Disease that spreads out of the language network leaves a non-verbal fingerprint specific to each variant. In logopenic PPA, visuospatial ability is impaired across the board. Non-fluent patients show executive and visuomotor deficits, and semantic patients show visual-memory deficits (Tee et al.). Processing speed is slowed in logopenic and non-fluent PPA but spared in semantic PPA (Gajardo-Vidal et al.). If these fingerprints are real, a patient whose non-verbal profile falls between two variant prototypes should be exactly the patient on whom clinicians split. A small probabilistic model should therefore predict the spread of expert opinion, not just the majority label. If the panel's spread also tracks mismatch between the clinical syndrome and the pathology found at autopsy, disagreement becomes a usable clinical warning.
Core Claim
The proposal makes two claims and tests a third. First, a low-capacity model of the panel's label distribution, trained on panel distributions rather than majority votes, predicts per-patient clinician disagreement in PPA. It does better than an equally simple classifier trained on consensus labels. Second, the predictive signal comes mostly from language-free measures: the three visuospatial components and the tablet "Match" speed score. It does not come mostly from the language tests that define the variants in the first place. Third, we test whether supplying clinicians with these language-free scores reduces their disagreement on mixed cases specifically.
Relation to Prior Work
The closest PPA work is the agentic GPT-4 system of Gallingani et al.. It simulates a multidisciplinary board with specialist language-model agents and reaches 90.7% accuracy across 15 syndromes, and 100/100/94.1% on the three variants, in 54 autopsy-confirmed cases. It motivates this proposal and also constrains it. The authors restricted the cohort to prototypical, congruent cases. Their system outputs a single label. Its reasoning was stable across 96% of cases over 10 reruns, which suggests that resampling the system gives almost no spread from which ambiguity could be read. We do the opposite: we target the non-prototypical cases and treat the distribution of opinions as the quantity to predict.
The method we borrow comes from pathology. The Latent Doctor Model uses a conditional variational autoencoder, a generative network with a learned random code. It encodes individual raters from sets of cases each rater has labeled. It reports a KL divergence of 0.461 to the six-pathologist label distribution on Gleason2019, and a Spearman correlation of up to 0.728 between its predictive entropy and the 20-pathologist panel's entropy on ISUP grading. We keep its central idea: train on distributions, model rater identity explicitly. We reject its capacity. PPA cohorts are around 50 to 150 patients, not thousands of image patches.
Two papers give a reason to prefer a small model. Ganz et al. found that a logistic regression on mitotic count matched a combined deep model for meningioma grading, at 32/43 versus 33/43 correct, and beat an image-only network (Spearman 0.79 versus 0.72). Tee et al. classified 79.7% of 118 patients from three visuospatial tests. They also warn that these tests are insufficient alone and partly verbal, which is why we add the fully non-verbal Match task. The Fraser et al. speech classifiers separate semantic from non-fluent PPA at 70–75%. They are purely linguistic and omit logopenic PPA, so they serve here as a language-based comparison feature set.
Research Program
The main product is a dataset. We will assemble de-identified case files from a memory-clinic cohort that collected the visuospatial battery and TabCAT Match; the UCSF Memory and Aging Center cohorts described in the file collected both. Selection will be stratified so that roughly half of the cases met consensus criteria cleanly and half were recorded as mixed or unclassifiable. An autopsy-confirmed subset will include the Gallingani et al. cases plus any non-prototypical autopsy cases available. Eight to ten clinicians from at least two centers will each assign every case a variant or "unclassifiable," together with a confidence rating.
The reading follows a crossover design with a washout period between sessions. In condition A, clinicians see the standard workup: history, language testing and MRI atrophy summary. In condition B, they see the same workup plus the language-free profile, shown as three visuospatial component scores and Match speed, each age-normed. Measuring the panel twice tests whether non-verbal information changes agreement. It also separates "the profile predicts disagreement" from "the profile would resolve disagreement."
The model is deliberately small. For patient features $x$, it outputs latent variant probabilities $\pi(x)$ through a multinomial logistic regression. Each rater $r$ is given a confusion matrix $C_r$ estimated with a shrinkage prior. $C_r$ captures rater habits, such as one clinician calling borderline cases logopenic more readily than the others. The probability that rater $r$ assigns label $y$ is then
$$p(y \mid x, r) = \sum_{c} \pi_c(x)\, C_r[c, y],$$
where $c$ ranges over the three variants and $C_r[c,y]$ is the probability that rater $r$ says $y$ when the latent variant is $c$. This is a scaled-down, patient-conditioned relative of the Latent Doctor rater encoding. It is fitted by maximum likelihood on all individual ratings with leave-one-patient-out evaluation. The panel distribution predicted for a held-out patient is the average over raters.
Baselines
The naive reference point predicts, for every patient, the marginal label frequencies observed across the panel. A model that cannot beat it in KL divergence to the held-out panel distribution has learned nothing patient-specific.
The strongest classical competitor is the posterior of a discriminant classifier trained on majority-vote labels, in the style of Tee et al.. Tee et al. report 79.7% variant accuracy from three visuospatial tests; this is a hard-label accuracy, not a disagreement metric. It is the most dangerous baseline, because its class posterior already yields an entropy for free. We will give it identical features, identical leave-one-patient-out splits and the same regularization search. Beating it means a lower KL divergence to the panel distribution and a higher Spearman correlation between predicted entropy and panel entropy, with confidence intervals from a patient-level bootstrap.
The strongest existing system is the Gallingani et al. agentic pipeline. Accuracy on the prototypical cohort is reported, at 90.7% open-ended and 100/100/94.1% across the three variants. No reported result was found on Lacuna for its agreement with a multi-rater panel or for its performance on unclassifiable cases. We will re-run it on our cases with the same inputs the clinicians receive. Its uncertainty will be scored two ways: from 10 resamples, and from directly elicited variant probabilities.
We also include a scaled Latent Doctor variational autoencoder trained on the same ratings. This tests whether its extra capacity helps or overfits at this sample size.
Two ablations isolate the proposal's key ingredients. The first removes the language-free features and keeps only the language and MRI features. The second keeps the features but trains on majority labels, with rater matrices fixed to the identity.
Why not just…?
Why not just use the discriminant classifier's posterior entropy? This is a genuine threat. If patients sit between prototypes, a hard-label classifier's posterior may already be flat there. The second ablation settles the question directly. If soft-label training does not beat the hard-label posterior in predicting panel entropy, we will report that. The contributions that remain would be the panel dataset, the language-free crossover result and the pathology analysis, and the modeling claim would be dropped.
Why not just apply the Latent Doctor Model as published? It was built for thousands of image patches and support sets of images per rater. With about 100 patients and 10 raters, a variational autoencoder has little data to learn its latent space. The Ganz et al. result suggests that simple models on meaningful quantities are competitive in small clinical cohorts. Including the scaled variational autoencoder as a baseline tests this rather than assuming it.
Why not just ask the language-model board for probabilities? Its reported run-to-run stability of 96% implies that resampled spread will be small. Elicited verbal probabilities from language models are not known to be calibrated to human panels. It was also never tested on mixed cases. We run it anyway. If its elicited probabilities match the panel as well as our model does, that is a useful, cheaper result.
Decisive Checks
The first decisive check is whether the full model beats the hard-label posterior on held-out KL divergence and entropy correlation, and whether removing the language-free features erases that advantage. If neither ablation changes the result, the core claim fails.
The second check compares Fleiss' kappa, a chance-corrected measure of agreement among many raters, between conditions A and B, separately for prototypical and mixed cases. The prediction is that kappa rises in condition B on mixed cases, while prototypical cases show little change.
The third check uses the autopsy subset. We test whether panel entropy, and separately model-predicted entropy, distinguish patients whose pathology is atypical for their majority variant, scored by area under the ROC curve. An example of atypical pathology is Alzheimer pathology in a non-fluent case, which Montembeault et al. describe as a common source of mismatch. This last test is underpowered at autopsy-cohort sizes and will be reported as an estimate with its interval, not as a confirmation.
References
- Clinical, Anatomical, and Pathological Features in the Three Variants of Primary Progressive Aphasia: A Review
- Neuroanatomical correlations of visuospatial processing in primary progressive aphasia
- Assessing processing speed and its neural correlates in the three variants of primary progressive aphasia with a non-verbal tablet-based task
- Agentic Generative Artificial Intelligence System for Classification of Pathology-Confirmed Primary Progressive Aphasia Variants
- The Latent Doctor Model for Modeling Inter-Observer Variability
- Automatic and explainable grading of meningiomas from histopathology images
- Automatic speech recognition in the diagnosis of primary progressive aphasia