Disagreement-Scaled Flow Matching for MRI Synthesis from Any Subset of Scans

Brain tumor MRI is normally acquired as four scan types, or sequences: T1, contrast-enhanced T1, T2, and FLAIR. In practice one or more is often missing. With four sequences there are 15 possible subsets of available scans, and a system that trains one model per subset does not scale. The fastest recent synthesizer, WFM: 3D Wavelet Flow Matching for Ultrafast Multi-Modal MRI Synthesis, produces a missing 3D volume in under a second. It uses flow matching (a generator that learns a straight path from a starting guess to the target image, so it needs only one or two network evaluations instead of dozens). It gets its speed by starting from an "informed prior": the average of the three other sequences, computed in a wavelet (multi-frequency) representation. That prior only makes sense when exactly one sequence is missing. When only T1 is available, the "average" is a T1 image, which is a poor starting point for FLAIR, and the straight one-or-two-step path breaks down.

We propose one model that synthesizes any missing sequence from any available subset while keeping WFM-level speed. Each available scan goes through a small encoder that predicts the target contrast. These predictions are pooled HeMIS-style into a per-voxel mean and a per-voxel variance (HeMIS). The flow starts from the pooled mean, with noise scaled by the pooled variance. Where the available scans agree, the generator barely moves. Where they disagree, which tends to happen at tumor boundaries and enhancing regions, it has room to generate, and it can spend a few extra steps there. The result should be fast, handle all 15 subsets, and come with an uncertainty map at no extra cost.

Core Claim

The claim is that a learned, variance-aware prior removes the main reason fast flow-matching synthesis fails outside the 3→1 setting. The rigid prior sits far from the target when few scans are present, and a straight path over a long distance needs many steps. We define the starting point as

$$x_0 = \hat\mu_S + \hat\sigma_S \odot \epsilon, \quad \epsilon \sim \mathcal{N}(0, I),$$

where $S$ is the set of available sequences. $\hat\mu_S$ and $\hat\sigma_S$ are the per-voxel mean and standard deviation of the target-contrast predictions made by the encoders of the scans in $S$, and $\odot$ is element-wise multiplication. When $|S|=1$ the variance cannot be estimated from disagreement, so a small learned head predicts $\hat\sigma_S$ directly.

The flow network learns the straight-path velocity from $x_0$ to the true target. The expected transport distance shrinks wherever the scans agree, so the number of integration steps can be chosen per volume from the mean of $\hat\sigma_S$ inside the brain. Few steps are used when many scans are present, and more when few are. Compute then grows with how much information is missing, rather than with the number of subsets.

What is new relative to WFM is threefold: arbitrary-subset conditioning, a learned pooled prior in place of a fixed average, and noise and step count tied to disagreement between scans. Relative to HeMIS and Hetero-Modal Variational Encoder-Decoder for Joint Modality Completion and Segmentation, which pool per-modality features to segment or to reconstruct in latent space, we use the pooled variance as the noise schedule of a few-step generator operating on real images.

Research Program

Architecture. The velocity network works in the 3D wavelet representation, as WFM does. It adds long-range context only at the coarsest wavelet band, where the grid is about one-eighth of the original size per axis. There we insert recurrent criss-cross attention, taken from CCNet: Criss-Cross Attention for Semantic Segmentation. CCNet makes each voxel attend along its row and column, then repeats the step once so information reaches the whole grid. It reports 11× less GPU memory and 85% fewer FLOPs than dense non-local attention. We extend it to three axes. The motivation is that contrast enhancement and edema extent depend on tumor-wide structure that small convolution kernels cannot see. Paying for attention only at the coarse band keeps memory low, whereas dense 3D attention as in mmFormer brings the model to 106M parameters.

Training. Each case is trained with a randomly dropped subset of sequences, the modality-dropout recipe mmFormer uses. The losses are the flow-matching loss plus a per-encoder prediction loss that trains the pooled mean.

Adaptive step count. A simple rule maps the mean pooled variance to one, two, or four steps. The rule is fit on validation data.

Edge-oriented checks. We report parameters, peak memory, and wall-clock time per volume. As a secondary test, we apply INT8 post-training quantization. Timestep-Aware Correction for Quantized Diffusion Models shows that per-step calibration is what makes low-bit diffusion work. With one to four steps, per-step calibration is cheap, and we test whether the variance-scaled input keeps activation ranges stable across subsets.

Baselines

Naive reference points. The weakest reference copies the most similar available sequence into the missing slot. The second is WFM's own prior, the plain average of available sequences, used as the output with no generation. Both are evaluated with the same downstream segmenter as every other method. HeMIS reports that a mean-filling baseline loses to its pooled representation in 14 of 15 missing-data configurations for tumor core, so we expect these to be clearly beatable. They set the floor.

Strongest fast competitor. This is WFM retrained on BraTS 2024 with modality dropout and its average-of-available prior, using the same wavelet backbone, data splits, epochs, and hyperparameter search budget as our model. It is the honest "obvious extension." WFM reports 26.8 dB PSNR and 0.94 SSIM at 0.16–0.64 seconds per volume with 82M parameters on BraTS 2024, but only for the 3→1 setting. We must match it there within about 0.5 dB and beat it when one or two scans are available.

Quality ceiling. cWDM is a many-step wavelet diffusion model, reported in the WFM paper at 28.4 dB PSNR, 0.95 SSIM, and 160 seconds per volume in the 3→1 setting. We run it where compute allows. We do not expect to beat it on PSNR. Closing a meaningful fraction of the gap at roughly 250× lower latency counts as success.

Direct segmentation. mmFormer segments directly from incomplete inputs without synthesizing anything. The file lists its +19.07% enhancing-tumor Dice gain over prior methods with a single modality, but no absolute Dice value on Lacuna.

Ablations. Four ablations isolate the key ingredient: pooled mean with fixed unit noise instead of variance-scaled noise; a fixed step count instead of the adaptive rule; the model without coarse-band criss-cross attention; and a deterministic one-pass regressor, described in the next section.

Why not just…?

Why not just regress the missing scan in one pass from the pooled features? This is the most serious threat. A one-step flow from a good prior is close to regression. CyIN shows that deterministic cascaded autoencoders, with feature purification, stay competitive while running 3–5× faster than iterative imputers. ADMC reports that deterministic completion falls from 82.7% to 75.2% accuracy at 50% missingness, but that result is on emotion features, not images. We train a regressor with identical encoders, backbone, and compute. We compare three things: downstream enhancing-tumor Dice, high-frequency fidelity inside the tumor, and whether sample spread predicts error. If the regressor matches us on all three, we will report that the variance-pooled prior, not the generator, carries the gain, and the contribution shrinks accordingly.

Why not just segment directly with mmFormer and skip synthesis? If only a tumor mask is needed, direct robust segmentation may win, and we will report it honestly. Synthesis matters for two reasons. Radiologists read images, not masks, and standard segmenters trained on complete data can be reused unchanged. This is the premise of the BraSyn challenge. We compare Dice from our synthesized scans fed to a complete-data segmenter against mmFormer on identical subsets.

Why not just train WFM with modality dropout? That is exactly our strongest baseline. The experiment below tests whether its fixed-average prior degrades at one or two available scans.

Proposed Experiments

We use BraTS 2024 with WFM's split of 1,032 training and 219 test volumes. For every non-empty proper subset of the four sequences, we synthesize each missing sequence and report PSNR and SSIM overall and inside the tumor mask.

Downstream quality follows the BraSyn protocol. A fixed segmenter trained on complete data receives real plus synthesized scans, and we report Dice for enhancing tumor, tumor core, and whole tumor. We also report the correlation between the per-voxel spread of five samples and the actual error, along with latency, peak memory, and parameters. The INT8 variant is tested on the same metrics.

Three decisive checks determine the outcome. First, with a single available scan, our model beats dropout-trained WFM on enhancing-tumor Dice, and the gap shrinks toward zero at three scans. That shrinking gap would confirm that the prior's distance to the target is the mechanism. Second, the variance-scaled model beats both the unit-noise ablation and the one-pass regressor on tumor-region Dice or on error calibration. Otherwise the generative part is unjustified. Third, average latency stays under one second per volume while spending measurably more steps on low-cardinality inputs. A model that needs four steps everywhere would mean the adaptive-compute claim has failed.