After radiotherapy or immunotherapy, a brain tumor can look larger on MRI because the treatment is working. Inflammation, leaky vessels and dead tissue swell the lesion, and the scan mimics real tumor growth. This is called pseudoprogression. Telling it apart from true progression decides whether a patient stays on a working therapy or moves quickly to salvage treatment. The current clinical fix is to wait: the immunotherapy response criteria (iRANO) require a confirmatory scan about three months later. In one glioblastoma cohort this wait delayed confirmation of true progression by 42 to 93 days in 4% of patients (Xin Chen et al.). Machine learning seems like the obvious answer, but the problem resists it for three reasons. Pseudoprogression is a rare class. Radiologists disagree about individual cases. Trustworthy labels need biopsy or months of follow-up, and nobody will hand-contour hundreds of post-treatment lesions.
We propose to stop treating this as a rare-patient classification problem and instead learn a voxel-by-voxel map of where viable tumor sits inside the enhancing lesion. The map is trained from two free and reader-independent signals. The first is a physics-based teacher: parametric response maps of water diffusion, which flag voxels where cells are tightly packed. The second is a weak patient-level constraint taken from the follow-up outcome: a lesion later adjudicated as true progression must contain a large viable fraction, and a pseudoprogression lesion must not. The patient-level decision then becomes a threshold on the predicted viable fraction.
The key observation concerns class imbalance. Pseudoprogression patients are rare, but treatment-effect voxels are not. Most progressing lesions also contain necrosis alongside viable tumor. At the voxel level the class imbalance largely disappears, and every patient contributes thousands of labeled examples of both tissue types.
Core Claim
Our central claim is this. A network trained on automatically segmented lesions, with diffusion response maps as a noisy dense teacher and follow-up outcomes as a bag-level proportion constraint, will distinguish pseudoprogression from true progression at the time of the first suspicious scan. It will do so as well as or better than supervised patient-level classifiers, without manual contours or reader labels. It will also stay calibrated at a realistic 10–20% pseudoprogression prevalence.
A secondary claim is that the learned map can be distilled into routine structural sequences (T1, T2, contrast-enhanced T1, FLAIR). The model would then remain usable at test time when diffusion imaging is missing, misregistered, or acquired with inconsistent settings. Inconsistent acquisition across scanners is a documented failure mode for cross-center models (Spyridon Bakas et al.).
Why this is the right combination
Each of the four directions supplies one role.
Pseudoprogression is the application, and its biology supplies the teacher. Caroline Reimer et al. computed relative apparent diffusion coefficient (ADC, a measure of how freely water moves, which is low where cells are densely packed). They normalized it to healthy white matter and registered follow-up scans to baseline. Voxels whose ADC dropped by at least 0.25 were called hypodiffusive. True progression lesions averaged 59.2% hypodiffusive volume versus 18.6% for pseudoprogression, giving an AUC of 0.844 on 35 patients. That study motivates our design and constrains it in three ways. It shows that a viable fraction separates the classes. It required manual 3D contours. It used a fixed threshold with no learning, on a 28-to-7 skewed cohort. Its maps are also noisy in known ways, including hemorrhage, susceptibility artifacts and misregistration. Reimer's work, together with Julian Ziegenfeuter et al., establishes that viable tumor and necrosis coexist inside one enhancing rim. That is why we predict a map rather than a single whole-lesion label.
Class imbalance is the constraint the design must survive. Gregory Holste et al. showed on long-tailed chest X-ray benchmarks that the classification head, not the image features, is what collapses on rare classes: tail accuracy was 1.7% under plain cross-entropy and 30.0% with balanced head retraining. They also found that MixUp-style image blending erases fine texture cues. That finding argues against using Balanced-MixUp on post-treatment MRI, even though it worked on retinal images. Our response is not to rebalance a rare patient class. We move supervision to a level where the classes are not rare, and keep patient-level balancing only in the weak constraint.
The annotation bottleneck supplies the method for the weak constraint. Takamasa Yamaguchi et al. trained frame-level ulcerative colitis severity scores using only the patient-level maximum recorded in the chart. Their loss penalizes any frame that exceeds that maximum. We reuse the idea that a chart-level fact constrains an aggregate of instance predictions, applied here to a viable-tissue proportion inside a 3D lesion. Lesion masks come from an automatic segmenter trained on the BraTS challenge data, so no expert contouring is needed.
Inter-observer variability is the evaluation lens, not the training target. The latent-variable model of Christian et al. reproduces a panel's label distribution. The per-rater ensembles of Jan Grove et al. improved calibration over consensus labels. Both are useful when readers are the best available truth. Christian et al.'s own stated limitation is that a panel sharing a systematic bias gets that bias reproduced. Here the truth is defined by later imaging and histology, so we keep readers out of the training labels. Instead, we test whether the model's uncertainty concentrates on the cases where independent readers disagree.
Research Program
We assemble post-treatment glioblastoma cohorts. The inclusion scan is the first follow-up MRI that radiologically suggests progression and includes DWI and a pre-treatment or early post-treatment baseline. Each patient carries an outcome label adjudicated from follow-up imaging, or from histopathology where re-resection occurred. Enhancing-lesion masks are produced by a BraTS-trained segmenter. The pre-operative UPenn-GBM cohort (Bakas et al.) supplies 630 multi-sequence cases for pretraining the segmenter and image encoder, but it contains no post-treatment follow-up. We will check whether the ACRIN-DSC-MR-Brain repository cited in Bakas et al. contains usable post-treatment diffusion and outcome data for an external test. If it does not, external testing uses a second institution.
The model is a 3D encoder-decoder that outputs, for each voxel $v$ in lesion $i$, a probability $p_v$ that the voxel is viable tumor. It is trained with
$$ \mathcal{L} = \sum_{v} c_v\,\mathrm{BCE}(p_v, m_v) \;+\; \lambda \sum_{i} \omega_{y_i}\,\max\!\big(0,\; s_i(\tau - \bar p_i) + \delta\big). $$
The terms are defined as follows:
- $m_v$ is the binary hypodiffusive label from the diffusion response map.
- $c_v$ is a confidence weight that down-weights voxels with susceptibility artifacts, large registration residuals, or ADC changes near the threshold.
- $\bar p_i$ is the mean predicted viability over lesion $i$.
- $s_i$ is $+1$ for adjudicated progression and $-1$ for pseudoprogression.
- $\tau$ and $\delta$ are a fraction threshold and margin, tuned within training folds.
- $\omega_{y_i}$ is an inverse class-frequency weight over patients, which is the only place class balancing enters.
- $\lambda$ trades off the two terms.
The first term teaches spatial structure. The second lets the outcome override the physics teacher where the diffusion map is wrong, for example in inflammatory hypercellularity from immune infiltrates, which also restricts diffusion.
We train two input variants. The first uses DWI and structural sequences. The second uses structural sequences only, with the diffusion map used purely as a training-time teacher. Both variants produce the patient score $\bar p_i$ and a per-patient uncertainty from Monte Carlo dropout.
Baselines
The naive reference point is the clinical default. We report the standard RANO call at the inclusion scan, which labels every enlarging lesion as progression. This gives perfect sensitivity and zero specificity for pseudoprogression. Alongside it, we report the iRANO waiting rule, scored both on accuracy and on days to correct decision, using the delays of 42 to 93 days reported by Xin Chen et al. as context.
The strongest non-learned competitor is automated parametric response mapping. This is Reimer's pipeline with the same fixed −0.25 relative ADC threshold, applied inside our automatic masks, with the patient cutoff chosen on training folds. Reimer et al. report AUC 0.844 with 86% sensitivity and 86% specificity using manual contours on 35 patients. Our full DWI-input model must beat automated mapping on the same patients, same masks and same folds. Otherwise the learned component adds nothing when DWI is present.
The strongest learned competitor is a patient-level classifier in the style of ERN-Net (Yang Gao et al.). ERN-Net is a lightweight CNN on T1, T2 and contrast-enhanced T1 crops. It reported AUC 0.958 and 90.3% accuracy against 62.8% for neurosurgeons, but on a balanced cohort in which every patient had undergone re-resection. We reimplement it on our crops with the same encoder pretraining and the same hyperparameter search budget. We run it three ways: plain cross-entropy, class-weighted cross-entropy, and balanced head retraining (cRT) following Gregory Holste et al.. We omit MixUp because of that paper's texture-destruction finding. Beating this baseline means a higher balanced accuracy, and a higher pseudoprogression recall at a fixed 90% sensitivity for progression, at natural prevalence. The comparison must use paired bootstrap confidence intervals over patients.
Two ablations isolate the key ingredient. The first removes the proportion term, giving a physics teacher only. The second removes the dense term, giving outcome-only learning from label proportions. If the full model does not beat both ablations, the claim that combining the two signals is what matters fails.
Why not just…?
Why not just threshold automated diffusion response maps and skip learning? This is the most serious threat. With good DWI and good registration, it may match our DWI model. The head-to-head baseline above settles that. If automated mapping wins or ties, we will report that the physics suffices and narrow the contribution to the structural-only variant. That variant addresses the settings where the map cannot be computed: missing DWI, mismatched b-values across centers, or no usable baseline scan. The cross-center experiment tests exactly those settings.
Why not just train a patient-level CNN with class rebalancing? Balancing fixes a biased classification head but not a starved representation. In the chest X-ray benchmark, even balanced retraining left tail accuracy at 30.0% with hundreds of tail images (Gregory Holste et al.). We expect a few dozen pseudoprogression patients. ERN-Net's strong numbers came from an artificially balanced surgical cohort, so they do not transfer to natural prevalence. The prevalence-controlled experiment below tests this directly by subsampling pseudoprogression cases.
Why not just train on consensus reads from several neuroradiologists? At the first suspicious scan, readers are guessing the future. A consensus label trains the model to copy that guess, including any shared bias toward calling progression. Consensus training also hurt calibration elsewhere: ECE was 0.183 versus 0.050 for per-rater panels in Jan Grove et al.. We include a consensus-read-trained variant of our model to measure this directly.
Proposed Experiments
The primary experiment uses patient-grouped cross-validation on the pooled development cohort at natural prevalence. It reports patient-level AUC, balanced accuracy, pseudoprogression recall at fixed progression sensitivity, and expected calibration error. Results are stratified by label source (follow-up versus histology), because follow-up-adjudicated labels carry the verification bias noted in the file.
A prevalence sweep subsamples pseudoprogression patients to prevalences from 20% down to 5%. It tracks how quickly each method degrades. We predict the voxel-supervised model degrades most slowly.
Cross-center transfer trains on one institution and tests on another. We compare the DWI and structural-only variants, since acquisition shift is the failure mode Bakas et al. highlight.
For the observer analysis, three to five neuroradiologists independently read each inclusion scan, blinded to outcome. We report their agreement, their accuracy against adjudicated outcome, and the Spearman correlation between model uncertainty and reader disagreement, with the 0.728 reported by Christian et al. as a reference from pathology. We also report reader accuracy with and without the viability map displayed.
A clinical-timing analysis counts patients whom the model would have correctly called at the inclusion scan, and sums the days earlier than iRANO confirmation.
For spatial validity, where stereotactic biopsy coordinates or re-resection specimens exist, we check predicted viability at sampled sites against histology. On held-out lesions we check agreement with the physics map. We also run a shortcut probe in the spirit of SpurCon: scores are recomputed after masking the resection cavity and scanner-specific regions.
The decisive checks are three. First, the full model must beat automated mapping when DWI is present. Second, it must beat the rebalanced patient-level CNN at 10% prevalence. Third, it must beat both single-signal ablations. If the first fails but the other two hold, the contribution is the structural-only variant. If the second fails, the voxel-level approach to imbalance is wrong.
References
- A Comparative Retrospective Study of Immunotherapy RANO Versus Standard RANO Criteria in Glioblastoma Patients Receiving Immune Checkpoint Inhibitor Therapy
- Artificial Intelligence for Response Assessment in Neuro Oncology (AI-RANO), part 2: recommendations for standardisation, validation, and good clinical practice
- Differentiation of pseudoprogression and real progression in glioblastoma using ADC parametric response maps
- Resolving spatial response heterogeneity in glioblastoma
- Long-Tailed Classification of Thorax Diseases on Chest X-Ray: A New Benchmark Study
- Balanced-MixUp for Highly Imbalanced Medical Image Classification
- Domain Adaptation for Ulcerative Colitis Severity Estimation Using Patient-Level Diagnoses
- The Latent Doctor Model for Modeling Inter-Observer Variability
- More is more: leveraging multi-rater information for whole slide images grading via virtual expert panel
- The University of Pennsylvania glioblastoma (UPenn-GBM) cohort: advanced MRI, clinical, genomics, & radiomics
- Deep Learning Methodology for Differentiating Glioma Recurrence From Radiation Necrosis Using Multimodal Magnetic Resonance Imaging: Algorithm Development and Validation
- SpurCon: Weighted Supervised Contrastive Learning for Mitigating Spurious Cues in Medical Imaging