Physics-Consistent Semi-Supervised Segmentation from Activatable MRI Contrast

When a tumor scan is acquired before and after a contrast agent is given, the difference between the two images is a free, label-free measurement of where the agent went and what it did. Smart contrast agents make that measurement unusually informative. A manganese-dioxide shell that dissolves only in the reducing, acidic tumor environment releases paramagnetic Mn²⁺ ions and brightens T1-weighted MRI precisely where the pathology is (A redox-activated theranostic nanoagent). Yet segmentation models trained with few expert labels ignore this physics. They either receive the post-contrast image as one more input channel or rely on generic heuristics such as prediction entropy to decide which unlabeled voxels to trust. We propose to treat the contrast change as a noisy physical observation of the tumor. A segmentation model's prediction on an unlabeled scan should, once passed through a small differentiable model of agent release and diffusion, reproduce the contrast change actually observed. The mismatch is a training loss that needs no annotation.

The opportunity matters on both sides. In semi-supervised segmentation, where a model learns from a few labeled scans plus many unlabeled ones, the hardest regions are thin, enhancing tumor margins. There even experts disagree: inter-rater Dice ranges from 74% to 85%, and the best algorithms reached only the high 50s to low 60s on enhancing core in BRATS. In pH- and redox-responsive nanomedicine, imaging is still read by hand from qualitative regions of interest. A model that segments well from a handful of labels, and that explicitly separates "tumor" from "where the carrier released," would turn activatable MRI into a quantitative per-voxel readout of release.

Core Claim

Our claim is that a forward-model consistency loss beats entropy-based consistency on enhancing-tumor boundaries at 5–10% labels. It should also beat simply giving the network the subtraction image as input. The loss requires that predicted tumor probability, mapped through a saturating release curve and a diffusion blur, explain the observed pre/post-contrast change. The gain should be largest when the contrast is activatable rather than always-on, because then the signal is specific to the microenvironment.

Concretely, let $s(x)\in[0,1]$ be the predicted probability that voxel $x$ is active tumor, and let $\Delta I(x)$ be the observed post-minus-pre T1 intensity change. The forward model predicts

$$\hat{\Delta I}(x) = a\,\big(G_\sigma * \rho(s)\big)(x) + b,$$

where $\rho$ is a monotone saturating release curve with learnable midpoint and steepness, $G_\sigma$ is a diffusion kernel of learnable width $\sigma$ applied by FFT, and $a,b$ are per-scan nuisance terms absorbing dose and relaxivity. The release curve encodes the carrier's switching behavior; the reported carrier is stable at physiological conditions and releases about 90% of payload at tumor-level glutathione. The diffusion kernel encodes leakage away from the release site. On unlabeled scans we minimize a robust (Huber) residual between $\hat{\Delta I}$ and $\Delta I$, gated by a perfusion mask. The gate exists because hypoperfused cores receive little agent and would otherwise be wrongly pushed toward "not tumor." Incomplete disassembly in hypoperfused regions is a known limitation of these carriers.

Relation to Prior Work

Three works motivate or supply parts of this design, and none does what we propose.

A redox-activated theranostic nanoagent builds a SiO₂@Au@MnO₂ particle whose shell dissolution simultaneously releases doxorubicin and switches on T1 contrast. It motivates our setting but performs no computational analysis. We borrow its release behavior as the shape prior for $\rho$. Acid-triggered systems behave the same way in principle: the ZIF-8 framework collapses when its imidazole ligands are protonated (Bi@ZIF-8). This suggests the same forward model should cover pH-activated carriers by changing only the release curve.

A Novel Domain Adaptation Framework also uses a physical model against data scarcity. It simulates tumor growth and mass effect with a partial differential equation, then uses a CycleGAN to make synthetic training images look real. The physics there generates extra labeled data and covers biomechanics only. We do the reverse: physics checks predictions on real unlabeled scans, and the physics is contrast-agent kinetics.

Variational reaction-diffusion systems shows that reaction-diffusion equations can be solved exactly with FFTs as differentiable layers. It reports a 408 ms forward solve at 511×255 on CPU. That work uses the equation as an abstract label smoother on 2D street and scene images. We reuse the FFT solver as a physical measurement model in 3D.

The semi-supervised methods we compete against all derive their unlabeled-data signal from the network itself. DyCON reweights consistency by voxel entropy over training. MisMatch enforces agreement between dilated and eroded feature maps. URPC uses disagreement across decoder scales as uncertainty. Our loss draws on an external measurement instead, and it can be added to any of them. MisMatch's argument that classes are not separated by low-density gaps at partial-volume borders is exactly why an external boundary signal is attractive.

The name "PH-Net" (PH-Net) refers to patch-wise hardness and is unrelated.

Research Program

The work proceeds in two stages, and the second is gated on the first.

Stage 1 uses clinical gadolinium MRI, which is always-on rather than activatable but comes with public pre/post pairs. BraTS provides T1 and contrast-enhanced T1 for every case, so $\Delta I$ = T1c − T1 after registration and intensity normalization. We train a 3D U-Net inside DyCON's mean-teacher setup. A mean teacher is a slowly averaged copy of the network that supplies targets for the student on unlabeled data. We add the forward-model loss on unlabeled cases. For gadolinium, $\rho$ is expected to collapse toward a near-linear leakage response, and we report the learned curve. This stage asks whether physical consistency helps at all when labels are scarce. It also tests the uncomfortable possibility that the enhancing label, being partly defined by enhancement, makes the gain trivial.

Stage 2 moves to activatable MRI with a small new preclinical dataset. Xenograft mice receive a Mn-releasing activatable agent of the kind in the theranostic nanoagent, and we image them before and after injection. Labels come from registered histology rather than from the MRI signal, so the label is no longer defined by the contrast. We compare against a passive (non-activatable) agent of matched size in the same model. The prediction is that the forward-model loss helps more with the activatable agent. With a passive agent, enhancement also reflects nonspecific leakage into healthy tissue, so the observation is a weaker proxy for tumor. As a by-product, the fitted $\rho(s)$ gives a per-voxel estimate of released fraction, which is the quantity nanomedicine studies currently eyeball. There is no public benchmark for this; no reported result was found on Lacuna for segmentation on activatable-contrast MRI.

Baselines

The naive reference points come first. The first is a supervised 3D U-Net trained on only the labeled fraction; the analogous from-scratch model in Positional Contrastive Learning reached Dice 0.184 on CHD with two labeled patients. The second is a "do nothing clever" physical baseline that thresholds $\Delta I$, with Otsu thresholding inside a brain mask, and takes the result as the enhancing-tumor segmentation. If thresholding matches our method, the network is not adding anything.

The strongest existing method is DyCON, because its entropy curriculum targets exactly the ambiguous boundary voxels we target. It reports Dice 61.48% versus 53.53% for BCP at 5% labels on ISLES'22 (DyCON). ISLES has no pre/post-contrast pairs, so our method cannot run there. On BraTS'19, DyCON reports gains over Mean Teacher, MCF, BCP and AC-MT, but no reported BraTS'19 number was found on Lacuna. We therefore rerun DyCON ourselves on identical splits.

For the fully supervised ceiling, the biophysics-augmented 2D model in A Novel Domain Adaptation Framework reports enhancing-tumor Dice 79.15% on BraTS'18. We treat this as context rather than a competitor, since it uses all 285 training cases.

We add MisMatch and URPC as secondary semi-supervised baselines. Every method uses the same backbone, the same labeled/unlabeled splits (5% and 10%, three seeds), the same co-registered T1 and T1c inputs, and an equal hyperparameter search budget of 20 trials on a held-out labeled validation set.

Two ablations isolate the key ingredient. The first is DyCON with $\Delta I$ appended as an input channel, which gives the network the same information without the physics. The second replaces the forward model with an identity link that regresses $s$ directly onto normalized $\Delta I$. That keeps the external signal but drops the saturation and diffusion terms. A further ablation removes the perfusion gate.

Beating the field means improving enhancing-tumor Dice and HD95 (the 95th-percentile surface distance) over DyCON-with-input-channel at 5% labels across seeds, with paired significance across cases. The improvement should be concentrated in a 3-voxel boundary band.

Why not just…?

Why not just feed the subtraction image to DyCON as an extra channel? This is the most serious threat. With enough labels, a network can learn whatever mapping from $\Delta I$ to label it needs. With 5% labels, an extra input is one more feature to overfit, and nothing ties the prediction on unlabeled scans to the measurement. The input-channel ablation runs on identical data and budget. If it matches our method in Stage 1, the physics adds nothing on gadolinium data, and the proposal's value rests entirely on Stage 2, where histology labels break the circularity.

Why not just threshold the contrast change and use it as pseudo-labels? Enhancement is not tumor. Vessels, choroid plexus and nonspecific uptake enhance, and necrotic or hypoperfused tumor does not. Expert agreement on these margins is already only 74–85% Dice (BRATS), which suggests no fixed threshold captures them. The Otsu baseline tests this directly. The forward model differs from a threshold in three ways: it lets the network explain enhancement through blur and saturation, it lets the network explain enhancement away through nuisance terms, and it applies only inside the perfusion gate.

Why not just add more unlabeled data or a stronger pretrained backbone? That is complementary rather than competing. Masked-autoencoder pretraining (Hi-End-MAE) improves the starting point but still provides no boundary-specific external signal. We will run the main comparison with and without such initialization. If pretraining erases our gain, we will report that.

Decisive Checks

Three results settle the question. First, on BraTS at 5% labels, does the full method beat both DyCON-with-input-channel and the identity-link ablation on boundary-band Dice and HD95? A gain over the input channel but not over the identity link would mean the external signal matters and the kinetic modeling does not.

Second, a counterfactual test: if we shuffle $\Delta I$ across patients, the gain must disappear. Otherwise the loss is acting as a generic regularizer rather than reading the physics.

Third, in Stage 2, the advantage over DyCON must be larger for the activatable agent than for the matched passive agent, measured against histology. If that gap is absent, microenvironment-triggered contrast offers no special leverage for label-scarce segmentation. That negative result would itself be worth reporting to both communities.