After radiotherapy for a brain tumor, a new enhancing lesion can be either returning tumor, which needs urgent treatment, or radiation necrosis, which is dead, inflamed tissue that usually settles on steroids. On routine MRI the two look the same, and clinicians over-call tumor to be safe. Functional measurements such as water diffusion (ADC, the apparent diffusion coefficient, a map of how freely water moves) and blood volume help separate them. But these maps are recorded on coarse voxels. At the lesion rim a single voxel averages fluid-filled cavity, healthy brain, dead tissue and a thin ribbon of viable tumor. This averaging, the partial volume effect, dilutes the tumor signal toward whatever surrounds it. A recurrence can then read as necrosis.
We propose to remove the healthy and fluid parts of each lesion voxel's signal before classifying. The contamination is estimated patient-specifically. We synthesize what that region would look like if healthy, using the patient's own opposite hemisphere warped into place, on the functional map itself and at sub-voxel tissue resolution. What remains after subtraction is an estimate of the abnormal tissue's own diffusion or perfusion value, with an uncertainty attached. We test whether that residual separates recurrence from necrosis better than raw lesion features, deep classifiers, or the standard contralateral ratio.
Core Claim
Part of the recurrence/necrosis mimicry is a measurement artifact, not biology. Specifically, it comes from mixing functional signal across the tissue interfaces of a post-surgical cavity. The literature flags this confound but does not correct it. The FET PET/MRI radiomics of Elder et al. applies no partial volume correction at lesion boundaries. Neither does the perfusion-then-FET triage of the sequential DSC/FET study, and neither do the ADC radiomics of Park et al..
The correction needs a "pure healthy tissue" value for each voxel, and population values are poor here. Du et al. showed in dementia perfusion that mis-modeling tissue fractions both exaggerates and hides disease signal. We therefore take the healthy value from the patient's contralateral hemisphere, which is the most specific normal control available. We deliberately synthesize only the healthy counterfactual and never the lesion. In healthy parenchyma, diffusion and perfusion are well predicted by anatomy. Inside lesions, structure and function uncouple, which is what makes MRI-to-PET synthesis ill-posed. Restricting the generator to the healthy counterfactual keeps the cross-modal step on the side where it is well-posed.
Research Program
Stage 1: a counterfactual healthy image. The first component is a 3D generator that produces a pseudo-healthy counterfactual of the lesion region in several contrasts: T1-weighted, FLAIR, ADC, and, where available, DSC relative cerebral blood volume.
It builds on BrainNormalizer, which inpaints tumor regions on 2D slices, guided by mirrored contralateral edge maps. We change three things.
First, conditioning is on continuous tissue-fraction maps rather than hard Canny edges. Rusak et al. showed that continuous partial-volume maps cut boundary-voxel error by 58.96% relative to binary segmentations. Sub-voxel interfaces are exactly what the unmixing step needs.
Second, the contralateral maps are brought across the midline by deformable registration, not rigid reflection. This handles mass effect and midline shift. A mild asymmetry prior keeps the generator from erasing benign left-right differences. Fineide et al. warn that over-constraining to strict symmetry creates false anomaly detections.
Third, the model is volumetric, using latent 3D diffusion in the manner of Make-A-Volume. Sampling it repeatedly gives a spread of plausible healthy counterfactuals, not a single point estimate.
The generator is trained on healthy-appearing tissue from BraTS2020 and non-lesional hemispheres. It is also trained on post-treatment scans with the lesion masked out.
Stage 2: unmixing each lesion voxel. For every functional voxel $v$ in the lesion region, the observed value $y_v$ is modeled as
$$y_v = \big[K * \big((1-a)\,c + a\,s\big)\big]_v + \epsilon_v,$$
where each term is defined as follows:
- $K$ is the scanner's blurring kernel at functional resolution.
- $c$ is the high-resolution counterfactual healthy-plus-fluid signal from Stage 1, already weighted by tissue and fluid fractions.
- $a$ is the high-resolution fraction of abnormal tissue, from the enhancing and FLAIR-abnormal masks at structural resolution.
- $s$ is the unknown pure abnormal-tissue signal, regularized to be spatially smooth.
- $\epsilon_v$ is noise.
Solving for $s$ is a regularized deconvolution with a subject-specific background. The structure resembles the supervised mixture unmixing of Asad et al. for spectroscopy. The difference is that there the pure signatures came from a population library, while here the background is patient-specific and synthesized.
Running the unmixing over many generator samples gives a distribution for $s$. The downstream classifier consumes the per-voxel posterior mean and variance of $s$ together with the abnormal fraction $a$. It then outputs a calibrated probability of recurrence, which responds to the clinical need for probabilities rather than brittle labels.
Baselines
Naive reference points. There are two. The first is a classifier on raw, uncorrected lesion ADC and structural features with the same architecture and training budget. The second is majority-class prediction, which matters because cohorts are dominated by recurrence (192 versus 42 in the ResNet10 study).
Contralateral-ratio normalization. This is the clinical standard. Lesion ADC or relative blood volume is divided by a hand-placed mirror region, as in the relative blood volume and tumor-to-background ratios of the sequential DSC/FET study, which reports a triage threshold of rCBVmax above 2.85. It uses the same inputs and the same downstream classifier. It is the closest competitor to the idea itself, because it already uses the opposite hemisphere; it simply does so with one scalar and no geometry.
Diffusion radiomics. Following Park et al., who report an external-validation AUC of 0.800 with ADC radiomics, we extract their feature classes from our segmentations.
End-to-end 3D CNNs. These are the strongest learned competitors: ERN-Net from Gao et al., which reports AUC 0.958 and specificity 81.7% on a 146-patient single-center cohort, and ResNet10 from the 234-patient study, which reports AUC 0.828 and specificity 0.939. Both are retrained on our splits with matched hyperparameter search, which is fairer than comparing to their reported numbers. We add a variant given the mirrored contralateral volume as an extra channel, in the spirit of the Siamese design of Tiulpin et al..
Ablations. These isolate the key ingredients. Each replaces one element of the method:
- the subject-specific counterfactual $c$, replaced with population tissue values;
- deformable warping, replaced with rigid mirroring;
- continuous fractions, replaced with binary masks;
- the posterior distribution, replaced with a single generator sample.
What counts as beating the baselines. The method must match the retrained CNNs on AUC and exceed them on necrosis sensitivity at a fixed 95% recurrence sensitivity. It must also beat contralateral-ratio normalization by a margin outside the bootstrap confidence interval on external data.
Why not just…?
Why not just normalize by a contralateral region, as clinicians already do? This is the real threat. If the scalar ratio captures most of the benefit, the synthesis machinery is unjustified. A scalar ratio cannot remove fluid and healthy contamination voxel by voxel at a cavity rim, where the mixing varies over millimeters. We expect the gap to appear mainly in thin-rim and large-cavity lesions. The prespecified analysis stratifies performance by cavity volume and median abnormal fraction $a$. If the ratio matches us in every stratum, we report that the geometry adds nothing.
Why not just give a large 3D CNN the mirrored hemisphere as an input channel? Networks like SEAN and Siamese designs can learn comparisons implicitly. But with about 40 necrosis cases per cohort they have little data from which to learn a deconvolution. Their very high single-center numbers may also not survive external testing, as T1-CE radiomics did not in Park et al.. The mirrored-channel CNN is run as a baseline, and the external-site comparison settles it.
Why not just synthesize FET PET from MRI and classify on that? Structure-to-metabolism mapping is weakest precisely in early treatment effects. Standard fidelity metrics also miss clinically decisive local errors, as the synthetic CT benchmark showed for dose. We therefore synthesize only healthy tissue and keep the lesion signal measured.
Why not classical partial volume correction with population tissue values? This is the population-value ablation. Du et al. suggests fixed tissue values bias results when tissue is abnormal. Irradiated but non-lesional white matter is likewise shifted from population norms.
Proposed Experiments
Recovery on known ground truth. The first check uses digital phantoms. Tumor patches with known ADC and blood volume are inserted into BraTS2020 brains at controlled rim thickness, cavity size and midline shift, then blurred to functional resolution. We measure how well $s$ is recovered as rim thickness falls below one functional voxel, comparing our unmixing with ratio normalization and population-value correction. The method should recover $s$ where the ratio collapses toward the background value. If it does not, the program stops here.
Generator quality. We report FID and SSIM on BraTS2020 against BrainNormalizer's reported FID 27.1 and SSIM 0.76. We also report counterfactual ADC error on held-out healthy hemispheres, stratified by midline shift.
Clinical classification. We use a retrospective, pathology- or follow-up-verified post-radiotherapy glioma cohort modeled on the Tiantan and Park designs, with at least one external site with a different scanner. Metrics are AUC, necrosis sensitivity at fixed recurrence sensitivity, and calibration via Brier score and expected calibration error. Where a site has FET PET, the same unmixing is applied to tumor-to-background ratios. Elder et al. is the relevant comparison point, but no reported AUC for it was found on Lacuna.
Failure modes. Performance is reported separately for bilateral or midline-crossing lesions, where the contralateral control is compromised in the way Chen and Liu describe. There, the posterior variance of $s$ should rise and the classifier should abstain, not guess.
References
- Combined FET PET/MRI radiomics differentiates radiation injury from recurrent brain metastasis
- Sequential implementation of DSC-MR perfusion and dynamic [18F]FET PET allows efficient differentiation of glioma progression from treatment-related changes
- Differentiation of Recurrent Glioblastoma from Radiation Necrosis Using Diffusion Radiomics: Machine Learning Model Development and External Validation
- Hypoperfusion in frontotemporal dementia and Alzheimer disease by arterial spin labeling MRI
- BrainNormalizer: Anatomy-Informed Pseudo-Healthy Brain Reconstruction from Tumor MRI via Edge-Guided ControlNet
- 3D Brain MRI GAN-Based Synthesis Conditioned on Partial Volume Maps
- Predicting an unstable tear film through artificial intelligence
- Make-A-Volume: Leveraging Latent Diffusion Models for Cross-Modality 3D Brain MRI Synthesis
- Supervised partial volume effect unmixing for brain tumor characterization using multi-voxel MR spectroscopic imaging
- Application of Deep Learning on MRI for Discriminating Glioma Recurrence from Radiation Necrosis: Algorithm Development and Validation
- Deep Learning Methodology for Differentiating Glioma Recurrence From Radiation Necrosis Using Multimodal Magnetic Resonance Imaging: Algorithm Development and Validation
- Automatic knee osteoarthritis diagnosis from plain radiographs:a deep learning-based approach
- Symmetry-Enhanced Attention Network for Acute Ischemic Infarct Segmentation with Non-Contrast CT Images
- A comprehensive comparative study of generative adversarial network architectures for synthetic computed tomography generation in the abdomen
- AsT: An Asymmetric-Sensitive Transformer for Osteonecrosis of the Femoral Head Detection (Student Abstract)