Tissue-Fraction Unmixing for Multi-Contrast MRI Super-Resolution That Does Not Erase Lesions

When one MRI contrast is acquired quickly at low resolution, a second contrast of the same patient, acquired at high resolution, can guide its reconstruction. The two share anatomy even though they differ in brightness. The known failure is that this guidance is too strong. Edges and textures that exist only in the reference get copied into the target, and a lesion visible only in the target (a tumor bright on T2 and faint on the reference) can be smoothed away because the reference says nothing is there. Current guided networks split each image into a "shared structure" part and a "contrast-specific" part. Both parts are unconstrained real-valued signals, so each can quietly cancel the other. Nothing in the model stops the unique part from subtracting a lesion that the shared part failed to explain.

We propose to reconstruct both contrasts from a single physically meaningful latent: for every voxel, the non-negative fractions of a small number of tissue types, which sum to one. Each contrast is then a tissue-specific brightness signature times these fractions. Anatomy shared across contrasts lives in the fractions. A lesion is represented as an extra tissue type whose signature is free in the target but unconstrained by the reference, so it cannot be cancelled away. We solve for the fractions with multiplicative updates, meaning iterations that rescale each value by a positive ratio rather than adding a gradient step. These keep the fractions exactly non-negative and on the sum-to-one constraint without clipping, and we unroll them into a trainable network. The same fraction map that produces the super-resolved image also serves as a tissue and lesion segmentation. Reconstruction and segmentation are two readouts of one latent rather than two competing heads, and that is the multi-task claim we test.

Core Claim

Cross-modal guidance leaks and erases pathology because the decomposition into shared and unique parts allows cancellation. A non-negative partial-volume decomposition with an explicit target-only tissue class removes that freedom. It should preserve target-only lesions better than unconstrained decompositions at equal image-wide PSNR. Because segmentation is read directly from the same fractions, joint training of super-resolution and segmentation should not need gradient-conflict surgery. The three directions play different roles here. Cross-modal guidance is the application. Multiplicative updates are the solver that makes the constrained latent exact and differentiable. Multi-task learning is the question of whether shared, interpretable variables align tasks by construction.

Relation to Prior Work

Three lines of work motivate the design. Coupled Dictionary Learning for Multi-contrast MRI Reconstruction and Multimodal Image Denoising based on Coupled Dictionary Learning separate patches into common, unique and noise parts to fight texture copying. Decomposition-Based Variational Network for Multi-Contrast MRI Super-Resolution and Reconstruction (MC-VarNet) builds on the observation that gradients align across contrasts while intensities do not, and unrolls half-quadratic splitting with dictionary layers and U-Net denoisers. All three use signed components solved by least squares or additive splitting. We keep their common/unique idea, but make the components non-negative tissue fractions so that cancellation is impossible. MRI Super-Resolution using Multi-Channel Total Variation supplies the explicit thick-slice forward model we reuse. That forward model is a non-negative linear operator, which is what makes multiplicative updates applicable. Its edge-coupling prior is the analytical baseline we must beat.

The solver comes from the multiplicative-update literature. Generalized Nonnegative Matrix Approximations with Bregman Divergences gives the gradient-split fixed-point rule. Locality Preserving Nonnegative Matrix Factorization shows how a neighbor-graph penalty enters that rule by splitting the graph term into positive and negative parts. We build that graph from the high-resolution reference instead of from data points, which gives a second, softer channel of structural guidance. Two papers constrain the design. On Trivial Solution and Scale Transfer Problems in Graph Regularized NMF shows that graph-regularized factorizations can be trivially minimized by shifting scale between factors. Our sum-to-one constraint on fractions is exactly the normalization that rules this out. Near-Universal Multiplicative Updates for Nonnegative Einsum Factorization shows majorization-minimization multiplicative updates beating automatic-differentiation gradient methods on speed and held-out loss for non-negative tensor models, but only on relational and trade tensors. We carry its derivation strategy to an imaging forward model.

On multi-task learning, JoJoNet jointly trains sampling, multi-contrast reconstruction and T2* mapping with ordinary backpropagation. RotoGrad rotates the shared representation per task to align task gradients. Multiplicative Sparse Feature Decomposition for Efficient Multi-View Multi-Task Learning (SPLIT) factors a linear multi-view, multi-task weight matrix multiplicatively. SPLIT is the closest bridge between multiplicative structure and multi-task learning, but it operates on linear models over tabular views, with no spatial structure. We differ from all three by placing the multiplicative, non-negative structure in the image latent itself. Tasks share variables with physical meaning rather than a generic feature vector.

Model

Let $x_c \in \mathbb{R}_+^N$ be the high-resolution image of contrast $c$ over $N$ voxels. Let $H \in \mathbb{R}_+^{K \times N}$ hold per-voxel fractions of $K$ tissue classes, with each column summing to one. Let $W \in \mathbb{R}_+^{C \times K}$ hold the brightness of each tissue in each of $C$ contrasts. We model $x_c = (WH)_c$. One tissue row is reserved as the "target-only" class. Its reference signature is left free and it carries a sparsity penalty, so it switches on only where the target data demand structure the reference cannot explain. Given a low-resolution target $y$ with thick-slice operator $A$ (non-negative), a high-resolution reference $r$, and a reference-derived affinity graph with Laplacian $L$, we minimize

$$ \mathcal{L}(W,H)=\tfrac12\|y-A(WH)_{t}\|^2+\tfrac{\lambda_r}{2}\|r-(WH)_{r}\|^2+\mu\,\mathrm{tr}(HLH^\top)+\gamma\,\mathrm{KL}(H\,\|\,P_\phi)+\rho\,\|H_{\text{lesion}}\|_1 , $$

where $t$ and $r$ index the target and reference contrasts, $P_\phi$ is a non-negative prior fraction map predicted by a small CNN with parameters $\phi$, and $\lambda_r,\mu,\gamma,\rho$ are weights. Each term splits into positive and negative gradient parts $[\nabla]^\pm$. Each unrolled layer applies

$$ H \leftarrow \mathrm{normalize}\!\left(H\odot\frac{[\nabla_H\mathcal{L}]^- +\epsilon}{[\nabla_H\mathcal{L}]^+ +\epsilon}\right), $$

with an analogous update for $W$. Here $\odot$ is element-wise product, "normalize" rescales each column of $H$ to sum to one, and $\epsilon>0$ keeps any fraction from reaching zero and freezing permanently. That zero-locking risk matters most for the lesion class. Twenty or so unrolled layers are trained end to end. The trained quantities are $\phi$, the per-layer weights, and an initializer for $W$. The loss is target super-resolution error plus a segmentation loss on $H$ where labels exist.

Research Program

The first stage builds the unrolled solver and checks that it behaves as an optimizer. Before any learning, the objective should decrease at every step. We will measure how often fractions sit near $\epsilon$. The second stage trains on IXI with proton-density as reference and T2 as ×4 target, matching the MC-VarNet setup. This establishes image-quality parity. The third stage moves to BraTS2018, where tumors appear differently across contrasts and tumor labels exist. There we train the multi-task version: target super-resolution plus tumor segmentation read from $H_{\text{lesion}}$ and the other fractions. The fourth stage is a controlled leakage study. We insert synthetic lesions into the target contrast only, and synthetic reference-only textures into the reference only. This measures erasure and copying directly rather than inferring them from PSNR. A final transfer check uses fastMRI knee with proton-density as reference and fat-suppressed proton-density as target. We score reconstructions inside the expert pathology boxes of fastMRI+. That paper argues that image-wide PSNR does not certify pathology preservation, which is the gap we target.

Baselines

The naive reference point is B-spline interpolation of the low-resolution target with no guidance. Brudfors et al. report roughly 30.73 dB PSNR for their multi-channel total-variation method against 29.83 dB for B-spline on IXI T1. That is a different contrast pairing, so we rerun both methods on our split rather than quote across setups. Multi-channel total variation is the strongest analytical baseline because it uses the same forward model and edge-coupling assumption without learning. We run it with its automatic noise and hyperparameter estimation.

The strongest learned competitor is MC-VarNet. On IXI proton-density-to-T2 ×4 super-resolution it reports 42.25 dB PSNR for MC-VarNet-L, 0.55 dB above its nearest competitor (Lei et al.). It is the right competitor because it is also an unrolled, decomposition-based guided method. It differs from ours precisely in using signed components and additive splitting. We retrain it on identical splits, with a matched parameter count and number of unrolled stages, and give it the same tuning budget. On IXI, "beating it" means staying within 0.3 dB of its PSNR. The claimed advantage lies elsewhere. On BraTS2018 and the synthetic-lesion study, we must show significantly higher PSNR and contrast-to-background ratio inside lesion masks, and lower texture-copy error in reference-only regions. On BraTS2018 and fastMRI+ there is no reported result on Lacuna for MC-VarNet or any guided method, so all numbers there come from our reruns.

For the multi-task question, the baseline is a shared-encoder network with a super-resolution head and a segmentation head. It is trained three ways: with swept scalar loss weights, with RotoGrad, and JoJoNet-style with cross-contrast feature propagation and a plain joint loss. The file gives no reported RotoGrad number on these datasets. We also include a channel-concatenation U-Net, the common default.

Two ablations isolate the key ingredients. The first keeps the identical unrolled architecture and objective but replaces multiplicative updates with projected or softplus-parameterized gradient steps, which isolates the solver. The second removes the sum-to-one constraint and the target-only class, keeping signed common and unique components inside the same unrolled network, which isolates the decomposition. The proposal wins if the second ablation loses lesion fidelity even when the first does not.

Why not just…?

Why not just parameterize an ordinary unrolled network with softplus outputs instead of multiplicative updates? This is a real threat. Softplus guarantees non-negativity without new solver machinery. Hood & Schein found multiplicative updates up to 90 times faster than gradient-based fitting, with less than half the test loss, but that was on non-imaging tensors and not inside a trained unrolled network. We therefore do not assume the advantage transfers. The first ablation settles it directly. If softplus-projected steps match on lesion fidelity, we report that the benefit comes from the non-cancelling simplex decomposition, not the update rule. The multiplicative form would then be kept only if it is faster or more stable per unrolled layer.

Why not just concatenate the two contrasts into a U-Net? Texture copying under concatenation is documented by Lei et al. and Song. On JoJoNet's data, a standard U-Net reached 30.94 dB against 33.93 dB for its cross-contrast propagation network (Zhao et al.). Our synthetic reference-only-texture test measures copying explicitly, so the concatenation baseline is judged on the failure we claim to fix, not only on PSNR.

Why not just use RotoGrad or tuned loss weights on a standard shared encoder? RotoGrad does fix both magnitude and directional conflict in generic multi-task vision. It may well close the segmentation gap. Our claim is narrower: when tasks read the same physical variables, conflict should be small without intervention. We measure cosine similarity between the super-resolution and segmentation gradients on shared parameters, in both our model and the shared-encoder baseline. If RotoGrad on a standard encoder matches our segmentation Dice and lesion fidelity, the multi-task part of the claim fails, and we will say so.

Why not just super-resolve first and segment afterwards? Sequential pipelines are simpler and avoid multi-task balancing altogether. The concern is not forgetting. The concern is that a reconstructor trained only on image-wide error has no incentive to keep lesions. We train a frozen super-resolution model followed by a segmenter, and compare its tumor Dice with our joint model at equal compute. A small gap would mean the joint readout adds little beyond the decomposition itself.

Proposed Experiments

On IXI proton-density-to-T2 ×4, we report PSNR and SSIM against all baselines, with parity to MC-VarNet as the bar. On BraTS2018 with T1 or FLAIR as reference and T2 as target, we report whole-image PSNR, lesion-mask PSNR, and lesion contrast-to-background ratio. We also report tumor Dice from the model's own fraction map and from a fixed segmenter applied to every method's reconstruction. That second Dice measures pathology preservation independently of our readout.

The synthetic study sweeps lesion size and contrast in the target only. It reports detection rate and residual lesion intensity, plus copied-texture energy in reference-only regions. On fastMRI with fastMRI+ boxes, we report box-restricted error and the fraction of boxes where lesion contrast drops below half its ground-truth value.

The decisive check is a specific pattern across the two ablations. At matched whole-image PSNR, the signed-decomposition ablation and MC-VarNet should lose clearly more lesion contrast than the full model on BraTS2018 and in the synthetic study. If this does not happen, the central claim that non-cancelling tissue fractions prevent pathology erasure is wrong.