Cancer treatments often kill tumor cells within days, but the tumor takes weeks or months to shrink visibly. Clinicians who judge response by tumor size, which is the current standard, therefore keep non-responding patients on toxic, ineffective drugs for months. Diffusion MRI can see some of the earlier signal. It measures how freely water moves in tissue, and water moves more freely once dying cells lose their membranes. The obstacle is that an early follow-up scan mixes two things: tissue that has disappeared, and tissue that remains but has changed. Current methods fold both into one number, such as "mean diffusion went up by 15%," and discard where in the tumor the change happened and what kind of tissue it happened in.
This note proposes a representation that pulls the two apart. Compare the distribution of diffusion measurements inside the tumor before treatment with the distribution after the first treatment cycle using unbalanced optimal transport, a method for matching two piles of mass that may differ in size. The matching then reports how much tissue vanished (shrinkage) and how the tissue that remained moved in diffusion space (functional change), and it needs no voxel-by-voxel registration of a deforming, shrinking breast. The resulting small set of "change habitats," groups of tissue that changed the same way, is fused with a few clinical variables through a small attention layer. In that layer, receptor subtype decides how much each kind of change counts. When the follow-up scan is missing, the model skips the change tokens.
Core Claim
Given baseline and early-treatment diffusion MRI, a registration-free decomposition of change into removed mass and transported mass will predict pathological complete response better than whole-tumor delta features. Pathological complete response (pCR) means no invasive tumor is left at surgery. The claim is that this gain survives two controls: the same classifier given the same clinical variables, and the strongest baseline-only model. It also predicts that the advantage should be largest when read as early decision-making: how many eventual non-responders can be flagged after one treatment cycle while missing very few responders.
The idea has three parts, and it matters which part carries the weight. The optimal transport decomposition is the new representation. Subtype-conditioned fusion is a small, motivated interaction. Missing-timepoint handling is a practical requirement. If an ablation shows the decomposition alone, fed to gradient-boosted trees, matches the full model, the contribution is the representation and the deep fusion should be dropped.
Why This Gap Is Real
The delayed-response literature supplies the motivation and one unresolved conflict. Early prediction of pathological complete response to neoadjuvant chemotherapy combining DCE-MRI and apparent diffusion coefficient values in breast Cancer found that baseline perfusion and diffusion parameters did not predict pCR in 119 patients. After two cycles, changes in wash-in rate, time-to-peak and mean ADC reached an AUC of 0.886. ADC, the apparent diffusion coefficient, is the standard per-voxel diffusion measure. Every parameter in that study is an average over a hand-drawn region.
Temporal radiomics for non-invasive preoperative prediction of pathologic complete response to neoadjuvant chemoimmunotherapy in non-small cell lung cancer computed post-minus-pre differences of CT texture features across 263 lung cancer patients. Those "delta" features beat baseline features, but external AUC fell to 0.78. The strongest delta predictor was a shape feature, which suggests the model was largely reading shrinkage.
On the other side, Cancer-Net BCa and Intratumoral and peritumoral radiomics for the pretreatment prediction of pathological complete response to neoadjuvant chemotherapy based on breast DCE-MRI argue that baseline scans suffice if the features are rich enough. Our experiments test this conflict directly rather than assuming one side.
Habitat imaging motivates looking at sub-regions but constrains how to do it. Fusion model integrating multi-sequence MRI radiomics and habitat imaging clusters voxels of a single-time scan into three groups with K-means, with the count fixed at three by hand. The habitat-only model reached AUC 0.775 and the full concatenated XGBoost model reached 0.913. Study on heterogeneity of vascularity and cellularity via multiparametric MRI habitat imaging in breast cancer links such sub-regions to vascularity and cellularity. The REMAP study argues that separating biological resistance from slow structural regression is the central clinical question. We define habitats by how tissue changed rather than how it looked at one time, and choose their number by held-out stability.
The fusion literature supplies methods but has only been tested on static survival tasks. HEALNet passes each modality into a shared latent array and skips missing ones. CA-MLIF and ME-Mamba target gigapixel slides and genomics; ME-Mamba uses optimal transport to align tissue patches with gene groups. TabMixer lets tabular embeddings condition video features. We borrow transport from ME-Mamba but apply it across time, not across modalities. We borrow HEALNet's skip rule and TabMixer's tabular conditioning. Their token counts do not apply here: our inputs are a few dozen habitat tokens, not thousands of patches, so efficiency is not our concern.
Research Program
For each patient we take every tumor voxel at baseline and at the first on-treatment scan. Each voxel becomes a feature vector $x$ of diffusion quantities: ADC, signal at each acquired b-value (the diffusion-weighting strengths of the scan), and the synthetic correlated diffusion signal of Cancer-Net BCa. Voxels also carry a distance-to-boundary coordinate, which extends into a peritumoral shell and replaces a fixed 3 mm halo. This gives two empirical distributions, $\mu_0$ at baseline and $\mu_1$ on treatment, whose total masses are the two tumor volumes. We solve
$$\pi^\star = \arg\min_{\pi \ge 0} \sum_{x,y} c(x,y)\,\pi(x,y) + \lambda\,\mathrm{KL}(\pi \mathbf{1}\,\|\,\mu_0) + \lambda\,\mathrm{KL}(\pi^\top \mathbf{1}\,\|\,\mu_1),$$
where $\pi(x,y)$ is how much baseline tissue of type $x$ becomes on-treatment tissue of type $y$. The cost $c$ is squared distance in standardized feature space, and $\lambda$ sets how expensive it is to create or destroy mass instead of moving it.
Baseline mass left unmatched is tissue that disappeared; per region of feature space, this is shrinkage. Matched mass comes with displacement vectors $y - x$, for example "high-cellularity core tissue whose ADC rose." We cluster the displacement vectors, weighted by transported mass, with a Gaussian mixture fit on the training fold only. We also cluster the destroyed mass by its baseline features. Each resulting habitat becomes a token carrying its mass fraction, mean displacement, and position relative to the boundary. $\lambda$ and the number of clusters are chosen by inner cross-validation. Because this is distribution matching rather than registration, it tolerates the large deformations of a shrinking breast tumor.
Fusion is kept deliberately small because cohorts have a few hundred patients. Clinical tokens (receptor subtype, treatment arm, age) act as queries in one or two cross-attention layers over the habitat tokens, followed by a logistic head. The subtype-as-query design has a specific motivation. Intratumoral and peritumoral radiomics for the pretreatment prediction of pathological complete response found subtype-specific AUCs of 0.83 and 0.93 against 0.74 for a pooled model, which says the imaging-to-response mapping depends on subtype. If the on-treatment scan is missing, only baseline habitat tokens are supplied, following HEALNet's skip rule. We train with random dropout of the change tokens so a single model serves both situations.
The primary data source is the ACRIN-6698 sub-study of the multi-center I-SPY 2 trial. Cancer-Net BCa used its baseline (T0) multi-b-value diffusion MRI with pCR labels for 253 patients. The trial's imaging is organized by timepoint. The file documents use of T0 only, so the first concrete task is to confirm how many patients have an early on-treatment scan and which clinical fields are released. The multi-site structure allows leave-site-out testing, which addresses the scanner-transfer failure seen in the lung study's external drop.
Baselines
The naive reference point is change in whole-tumor mean ADC, fed alone and then with clinical variables into logistic regression. This is the kind of region-averaged diffusion change Early prediction of pathological complete response used, without its perfusion terms. We also include tumor volume change alone, which is the size-based standard that the whole field is trying to beat.
The strongest tabular pipeline reproduces Temporal radiomics and Fusion model integrating multi-sequence MRI radiomics and habitat imaging on our data. It uses PyRadiomics baseline, on-treatment and delta features, plus single-time K-means habitat features and the same clinical variables, with LASSO selection and both logistic regression and XGBoost. This is the most likely competitor, because at a few hundred patients tree ensembles on engineered features are hard to beat. It gets the same folds, the same clinical inputs, and a hyperparameter search at least as large as ours. No reported result was found on Lacuna for delta radiomics on ACRIN-6698, so we must produce this number ourselves.
The strongest baseline-only model is the volumetric ResNet on synthetic correlated diffusion from Cancer-Net BCa. That paper reports 87.75% pCR accuracy on ACRIN-6698 T0, compared with 79.84% for ADC maps and 84.58% for b=800 diffusion images. No reported AUC was found on Lacuna for this dataset, and accuracy depends heavily on class balance. We therefore retrain it on our splits and report AUC and balanced accuracy alongside accuracy.
The key ablation feeds our transport-derived habitat features into the same XGBoost used for the tabular baseline, with no attention. Beating the tabular pipeline with this ablation shows the representation works. Beating the ablation with the full model shows the subtype-conditioned fusion adds something.
A second ablation uses balanced transport, forcing all mass to move. This isolates whether the shrinkage/functional split, rather than the matching itself, carries the signal.
Beating a baseline means a paired, bootstrapped AUC improvement whose 95% interval excludes zero under leave-site-out evaluation. It must also show a higher non-responder detection rate at 90% sensitivity for pCR, with calibration no worse.
Why not just…?
Why not just feed delta radiomics and clinical variables to XGBoost? This is the real threat. With about 250 patients it may match any learned fusion. The proposal is built so this outcome is informative rather than fatal. The XGBoost-on-transport-features ablation tells us whether the gain lives in the representation. If whole-tumor delta radiomics under the same XGBoost matches our transport features, the core claim is falsified. The lung study's shift to a shape feature as top predictor, and its external drop to 0.78 (Temporal radiomics), suggest delta features mostly encode shrinkage. That is exactly the component our decomposition separates out, but only the experiment can settle it.
Why not just use a better baseline scan and skip the follow-up? Cancer-Net BCa reports 87.75% accuracy from T0 alone, so this alternative is serious. Early prediction of pathological complete response found baseline parameters uninformative, but it used region means rather than volumetric deep features. We run both conditions under one model on one cohort: our model with baseline tokens only versus baseline plus change tokens, and the retrained Cancer-Net model versus the same model given both timepoints stacked. If the follow-up adds nothing over a strong baseline model, that is a clinically useful negative result: extra scans would not be justified.
Why not just train a 3D network end to end on both timepoints? This is the natural deep-learning answer. With a few hundred labels and scanner variation across sites, it is likely to fit site artifacts. Its implicit registration also fails on tumors that shrink by more than half. We include the stacked-timepoint Cancer-Net variant as this test. If it wins under leave-site-out evaluation, the handcrafted transport step is unnecessary.
Why not just use HEALNet or CA-MLIF directly? Both were designed to pass thousands of patch or genomic tokens through attention. Here the imaging input reduces to tens of habitat tokens, so their efficiency machinery buys nothing. We reuse HEALNet's skip-on-missing rule, and a HEALNet latent bottleneck over our tokens is run as a variant of the fusion head. The question these methods do not address is what the tokens should represent across time.
Decisive Checks
The main result is leave-site-out AUC, balanced accuracy, and calibration for pCR on ACRIN-6698, comparing our model with every baseline above. The second headline is an early-decision operating point: the fraction of eventual non-responders correctly flagged after the first on-treatment scan while keeping pCR sensitivity at 90%. This is reported beside the same quantity from volume change alone, which translates the accuracy gain into patients who could switch therapy earlier.
A mechanistic check tests whether the decomposition means what we claim. Removed mass should correlate with volume change. Functional displacement toward higher ADC in retained tissue should carry predictive signal after conditioning on volume change, tested by a likelihood-ratio comparison of logistic models. Shuffling displacement vectors across patients while keeping mass fractions should destroy that signal.
A subtype check tests whether the learned subtype-dependent weighting reproduces the direction of the subtype-specific performance gap reported in Intratumoral and peritumoral radiomics. A robustness check removes the on-treatment scan from a random half of test patients. It confirms the model degrades gracefully to its baseline-only performance rather than below it.
The proposal fails cleanly if whole-tumor delta features under XGBoost match the transport features, or if the follow-up scan adds nothing over the strongest baseline-only model. Both are checked directly above.
References
- Early prediction of pathological complete response to neoadjuvant chemotherapy combining DCE-MRI and apparent diffusion coefficient values in breast Cancer
- Temporal radiomics for non-invasive preoperative prediction of pathologic complete response to neoadjuvant chemoimmunotherapy in non-small cell lung cancer
- Enhancing Clinical Support for Breast Cancer with Deep Learning Models using Synthetic Correlated Diffusion Imaging
- Intratumoral and peritumoral radiomics for the pretreatment prediction of pathological complete response to neoadjuvant chemotherapy based on breast DCE-MRI
- Fusion model integrating multi-sequence MRI radiomics and habitat imaging for predicting pathological complete response in breast cancer treated with neoadjuvant therapy
- Study on heterogeneity of vascularity and cellularity via multiparametric MRI habitat imaging in breast cancer
- Evaluation of treatment response and resistance in metastatic renal cell cancer (mRCC) using integrated 18F–Fluorodeoxyglucose (18F–FDG) positron emission tomography/magnetic resonance imaging (PET/MRI); The REMAP study
- HEALNet: Multimodal Fusion for Heterogeneous Biomedical Data
- CA-MLIF: Cross-Attention and Multimodal Low-Rank Interaction Fusion Framework for Tumor Prognostic Prediction
- ME-Mamba: Multi-Expert Mamba with efficient knowledge capture and fusion for multimodal survival analysis
- TabMixer: Noninvasive Estimation of the Mean Pulmonary Artery Pressure via Imaging and Tabular Data Mixing