Reconstruction-Aware Clinical-Radiomic Fusion for Accelerated MRI

Hospitals increasingly shorten MRI scans by recording only a fraction of the raw measurements and letting a neural network fill in the rest. Separately, many diagnostic models combine "radiomic" features with a handful of clinical variables such as age or serum markers. Radiomic features are hand-defined texture and shape statistics computed from the image. These two practices are now colliding without anyone checking the joint. The fine textures that radiomic features measure are exactly what an accelerated reconstruction is most likely to invent or erase. Every hospital will also run a different scanner, acceleration factor, and vendor reconstruction. So the reconstruction pipeline becomes a new, hidden source of site-to-site variation, sitting upstream of models that were validated on slow, fully measured scans.

We propose to make the fusion model aware of how much it can trust each imaging feature for each patient. For every accelerated scan, we generate several plausible reconstructions. We do this by re-solving from random subsets of the measured data and by running an ensemble of reconstruction networks trained on different source data. We then measure how much each radiomic feature varies across them. The fusion model uses this per-patient spread to pull unreliable imaging features toward the values the patient's clinical variables would predict. When the reconstruction is shaky, as it should be on an unfamiliar scanner, the model leans on the clinical data, which travels across hospitals better. When the reconstruction is stable, the model uses the full imaging signal. The claim to test is concrete: under a held-out deployment condition, such as an unseen reconstructor, acceleration factor, or coil setup, this fusion should lose less discrimination and stay better calibrated than a fusion model simply trained on accelerated images. In-condition, it should lose nothing.

What the literature already shows, and what it leaves open

Three groups of papers motivate the gap. Learning Task-Specific Strategies for Accelerated MRI (TACKLE) trains the sampling mask, an unrolled reconstruction network, and a task head end to end. It shows that reconstructions tuned for diagnosis can have lower pixel fidelity yet better task accuracy. Its predictors are image-only, it uses no clinical variables, and it does not test downstream metrics across institutions. Robustness of Deep Learning for Accelerated MRI: Benefits of Diverse Training Data shows that training reconstruction networks on many scanners and anatomies improves performance on unseen vendors at no in-distribution cost. It also shows that longer training raises in-distribution image quality while out-of-distribution quality collapses. All of its evidence is in image metrics such as structural similarity, never in downstream features. Ultra-fast MRI for brain-age prediction in a real-world cognitive disorders clinic gives the most direct real-world warning. A roughly 1.5-minute accelerated scan agreed well with the standard scan overall, but it shifted brain-age estimates in pipeline-specific directions and interacted with diagnosis for two pipelines, all at a single hospital. Together these say that reconstruction choices move downstream numbers in ways image quality scores do not reveal.

A second group constrains the design. Multi-center evaluation of radiomics and deep learning to stratify malignancy risk of IPMNs found that 3D radiomic models overfit across seven centers because slice thickness varied, while 2D features held up. That is direct evidence that acquisition differences, not biology, can dominate radiomic features. Towards quantitative intensity analysis of conventional T1-weighted images in multiple sclerosis shows that whole-brain z-scoring washes out biological differences, whereas scaling to an external reference tissue preserves them. We therefore will not rely on naive intensity normalization to absorb reconstruction effects.

A third group supplies the fusion setting and its competitors. Predicting microvascular invasion in solitary hepatocellular carcinoma fuses clinical markers, radiologist reads, and radiomics with a Transformer, and it holds AUC 0.94 to 0.96 across two external hospitals on conventional full-length scans. Radiomics-Based Machine Learning Classification Strategy for Characterization of Hepatocellular Carcinoma on Contrast-Enhanced Ultrasound shows that imaging carries most of the signal: clinical-only AUC was 0.698, radiomics 0.903, and combined 0.912. Deep learning-based super-resolution US radiomics to differentiate testicular seminoma and non-seminoma is the closest cross-over. An image-enhancement network raised external radiomics AUC from 0.69 to 0.91, and clinical fusion reached 0.97. However, it uses ultrasound and an unconstrained image-to-image generator. It never asks whether enhanced textures are real or how much to trust them per patient, which is the question here.

Core Claim

Treat each radiomic feature from an accelerated scan as a noisy measurement of the feature the fully measured scan would have given. Estimate the noise per patient from the reconstruction process itself. Then correct the feature before fusion with a classic regression-calibration step. Let $c$ be the patient's clinical variables and $\tilde{x}$ the radiomic feature vector from the accelerated reconstruction. Let $\mu(c)$ and $\Lambda(c)$ be the mean and covariance of fully-sampled radiomic features given those clinical variables, both learned on training data. Let $\Sigma$ be the patient-specific covariance of features across plausible reconstructions. The corrected feature is

$$\hat{x} = \mu(c) + \Lambda(c)\,\big(\Lambda(c) + \Sigma\big)^{-1}\big(\tilde{x} - \mu(c)\big),$$

and $\hat{x}$, together with $c$, enters an ordinary fusion classifier. When $\Sigma$ is small, $\hat{x}\approx\tilde{x}$. When a reconstruction is unreliable for some feature, that feature moves toward what the clinical data imply, so the classifier's evidence shifts toward the clinical variables without being retrained. This form comes from the conditional expectation of a Gaussian latent variable observed with Gaussian noise. We use it as a transparent, auditable estimator, not as a claim that features are Gaussian.

What is new is not the shrinkage formula. It is three things together. First, deriving $\Sigma$ from the physics of the specific accelerated acquisition. Second, conditioning the shrinkage target on clinical data. Third, evaluating whether this buys robustness across deployment conditions, where reconstruction-induced shift is the dominant threat.

Research Program

The first component estimates reconstruction uncertainty. For each scan we produce K reconstructions in two ways. The first is a measurement bootstrap: an end-to-end variational network re-solves from random subsets of the actually measured raw-data lines, a few percent held out each time, with the fully sampled center kept. The second is a reconstructor ensemble: variational networks trained on single-source data versus the diverse multi-dataset pool described in Robustness of Deep Learning for Accelerated MRI, plus a total-variation compressed-sensing solver. The bootstrap captures within-patient ambiguity. Ensemble disagreement is meant to grow on unfamiliar scanners, which is precisely the cross-site signal we want. Radiomic features come from 2D extraction with fixed bin widths, following the IPMN finding, and use reference-tissue intensity scaling where an appropriate tissue exists, following the multiple sclerosis study.

The second component checks the uncertainty itself. Before any classifier is trained, we ask whether the predicted per-patient spread $\Sigma$ actually tracks the true feature error $|\tilde{x}-x_{\text{full}}|$. We measure this by rank correlation across patients and features, and by coverage of nominal intervals. If it does not, the method has no foundation, and the project becomes a measurement study of reconstruction-induced radiomic drift. That would still be a useful result.

The third component is the corrected fusion model, a regularized logistic model on $(\hat{x}, c)$. We also run a decision-level variant mirroring the IPMN study's fusion, so results do not hinge on one architecture.

Data and the honest limits of "multi-center"

No dataset in our file combines raw multi-site scanner data with clinical labels. The multi-center clinical cohorts (hepatocellular carcinoma, IPMN, testicular tumors) contain only reconstructed images. We therefore define a "site" as a deployment condition: scanner vendor and field strength, coil count, acceleration factor, and reconstruction method. We evaluate leave-condition-out, mirroring leave-center-out practice. This is realistic, because a hospital adopting accelerated MRI will typically adopt its vendor's reconstruction, but it is not a substitute for true multi-hospital validation. We say so explicitly.

The study uses three resources. FastMRI Prostate supplies real raw multi-coil prostate data from 312 patients with slice-level PI-RADS scores, so undersampling artifacts are genuine rather than simulated. The file lists no serum markers for this dataset. If released metadata do not include age or PSA, we use gland volume from the fully sampled image as the non-texture covariate, and we report that this weakens the clinical arm. UPenn-GBM supplies 630 glioblastoma patients with multi-sequence MRI, tumor subregions, and IDH status for 515 cases. Age is a strong clinical predictor of IDH status, which makes this a good test of shrinkage toward clinical data. Raw data must be simulated from magnitude images using synthetic coil maps and smooth phase, so results here carry an "inverse crime" caveat: simulated data can be unrealistically easy to reconstruct. fastMRI brain data and the multi-vendor CC-359 brain dataset used in Robustness of Deep Learning for Accelerated MRI have no diagnostic labels. They serve the drift atlas: how much radiomic features move across vendors and reconstructors, measured against the fully sampled reference.

Baselines

The naive reference points come first. A clinical-only model shows what imaging adds at all. A fusion model trained and tested on fully sampled images gives an upper bound that no accelerated method should exceed. The "do nothing" deployment baseline is that fully sampled fusion model applied unchanged to accelerated reconstructions, which is what happens when a validated model meets a newly accelerated protocol. All three use the same features, patients, and cross-validation splits as the proposal.

The strongest practical competitor is acceleration-augmented fusion: the same feature set and classifier, trained on reconstructions pooled across many acceleration factors and reconstructors. It learns average robustness without any per-patient uncertainty. It gets the identical pool of training reconstructions the proposal uses to estimate $\Sigma$, so the two methods see the same images and differ only in how they use them. Hyperparameters for both are tuned with the same budget on training conditions only.

The second strong competitor is a TACKLE-style end-to-end image classifier (Learning Task-Specific Strategies for Accelerated MRI) with the clinical variables concatenated before its final layer. Its strongest reported number in our file is 91.36% classification accuracy on BraTS brain tumor detection at 64x acceleration. That figure is image-only and single-benchmark. We found no reported result on Lacuna for IDH prediction on UPenn-GBM, or for PI-RADS classification on FastMRI Prostate, under any accelerated reconstruction. The baseline numbers on those tasks will therefore be produced in this study. As a cross-site reference for what radiomics fusion achieves when centers truly differ, the IPMN study reports 74.3% AUC in cross-validation falling to 69.2% on an unseen center.

Two ablations isolate the key ingredient. The first replaces per-patient $\Sigma$ with a single global noise level per feature, estimated across the training set. This tests whether per-patient uncertainty matters or only feature-level reliability does. The second replaces the clinically conditioned target $\mu(c)$ with the population mean. This tests whether clinical conditioning, rather than generic shrinkage, drives the gain.

The proposal beats the strongest baseline only if two things hold. Its AUC drop from training conditions to held-out conditions must be at least 0.03 smaller than that of acceleration-augmented fusion, as a pre-registered margin. And its calibration, measured by Brier score and expected calibration error, must be better on held-out conditions. In-condition AUC must also stay within 0.01, with confidence intervals from patient-level bootstrap. A gain that appears only on simulated UPenn-GBM data but not on real FastMRI Prostate raw data does not count.

Why not just…?

Why not just train the fusion model on accelerated images and let it learn robustness? This is the most serious threat, and it may work as well. If the held-out conditions resemble the training mixture, augmentation alone could match us. Our argument is that augmentation learns one average reliability per feature, while the reliability of a given texture on a given patient depends on lesion size, location relative to coil sensitivity, and how far the scanner is from the reconstructor's training data. The ensemble-disagreement term is designed to grow exactly in that last case. The head-to-head comparison above settles it. If augmentation matches us on held-out conditions, we will report that the simpler method suffices.

Why not just use a reconstruction network trained on diverse data, which already generalizes across vendors? Robustness of Deep Learning for Accelerated MRI shows diverse training nearly closes the image-quality gap on CC-359. But near-equal image quality does not imply equal texture statistics. The brain-age study found pipeline-specific offsets despite agreement above 0.90 (Ultra-fast MRI for brain-age prediction). The drift atlas tests this directly. If radiomic drift under the diverse reconstructor is negligible relative to biological between-patient spread, the problem we target largely disappears, and we will say so.

Why not just drop unstable features, as radiomics studies already do with reliability coefficients? Global filtering is cheap and it helped in the IPMN study, where restricting to 2D features improved cross-center stability. Its weakness is that it discards a feature everywhere, even when the feature is reliable for most patients, and that it cannot react to a new site. The global-noise ablation is essentially this method in a softer form, and we also run hard filtering at matched feature counts.

Why not just train everything end to end, sampling included? TACKLE shows this works for image-only tasks. But our cohorts number in the hundreds of patients, end-to-end mask learning optimizes for the training site's distribution, and the result is a black box. The corrected fusion keeps a feature-level audit trail, the explainability concern raised for the hepatocellular carcinoma Transformer. We include the end-to-end model as a baseline rather than dismissing it.

Decisive Checks

The project turns on three results, each falsifiable. First, predicted per-patient feature uncertainty must correlate with true feature error against fully sampled references. We pre-specify a rank correlation of at least 0.4 on held-out conditions, both on real FastMRI Prostate raw data and on CC-359 cross-vendor reconstructions. Second, on leave-condition-out evaluation, the corrected fusion must meet the margins in the Baselines section against acceleration-augmented fusion, and the per-patient and clinically conditioned components must each contribute in the ablations. Third, the drift atlas must show that reconstruction choice moves radiomic features by a non-trivial fraction of between-patient variance, even under a diverse-data reconstructor. Otherwise the problem is smaller than we claim.

If all three hold, the work will have shown how to use accelerated MRI with existing clinical-radiomic models, and why per-patient reconstruction uncertainty belongs inside them. It would also motivate the true multi-hospital raw-data collection that no public resource yet provides.