Vessel2Brain: Edge-Aware Multi-Scale Flow Matching for 3D Brain Image Generation from Vessel Masks

Zhi Chen, Tianyang Zhang, Theodoros N. Arvanitis, Le ZhangarXiv 2026 · arXiv

TL;DR

Vessel2Brain is a novel generative framework that synthesizes high-fidelity 3D brain MRI volumes using only sparse vascular masks as input. By leveraging conditional flow matching with edge-aware multi-scale supervision, the model successfully "hallucinates" complex brain anatomy from minimal structural cues, outperforming traditional GAN and diffusion-based methods.

What Problem Does This Paper Address?

Generating realistic 3D medical images is a cornerstone of modern neuroimaging, useful for everything from data augmentation to surgical simulation. However, acquiring well-annotated 3D MRI data is expensive and time-consuming. In contrast, vascular structures (the "plumbing" of the brain) are much easier to obtain via dedicated angiography or automated segmentation.

The challenge lies in the sparsity. Vascular masks cover only a tiny fraction of the total brain volume and provide zero direct information about the surrounding gray or white matter intensities. Most conditional generative models (like Pix2Pix or standard Diffusion) are designed for "dense" conditions—where the input provides a rough sketch of the entire output. When faced with the extreme sparsity of a vessel mask, these models often struggle to maintain anatomical coherence or simply fail to generalize.

Methods

The authors propose Vessel2Brain, which shifts away from the popular diffusion paradigm toward Conditional Flow Matching (CFM).

The Flow Matching Advantage

Unlike diffusion models that iteratively denoise a sample, CFM learns a deterministic velocity field that transforms a simple noise distribution into the target brain image along a linear interpolation path. This approach offers better training stability and reduced sampling complexity, which is vital when the conditioning signal is so limited.

Key Architectural Innovations

  • UNet-style FlowNet: A backbone that uses Feature-wise Linear Modulation (FiLM) to inject continuous timestep embeddings, allowing the network to adapt its predictions across the flow trajectory.
  • Multi-Scale Supervision: The framework employs a main velocity head for full-resolution details and an auxiliary head that predicts velocity at a coarser resolution. This "global-to-local" strategy ensures the model captures large-scale anatomical coherence before refining fine-grained tissue textures.
  • Edge-Aware Objective: To ensure the synthesized tissue aligns perfectly with the vascular "skeleton," the authors use Sobel operators to create an edge weighting map. This forces the loss function to pay extra attention to vascular boundaries during training.
Fig. 1. Overview of the proposed Vessel2Brain framework. Given a 3D vascular label map and a time step t, a UNet-style FlowNet conditioned on timestep and label embed- dings learns a velocity field for conditional flow matching. Velocity fields are predicted at multiple spatial resolutions through a main velocity head and an auxiliary velocity head. Edge-aware weighting derived from Sobel operators on the vascular volume em- phasises vessel boundaries during training. The predicted velocity fields are integrated to generate the final 3D brain MRI volume, while auxiliary outputs provide additional multi-scale supervision.

Datasets

The model was evaluated on two public 3D brain MRI datasets with paired vascular annotations:

  1. CereVessMRA: A dataset providing high-resolution MRA and paired structural MRI.
  2. TopBrain Challenge: A more heterogeneous dataset used to test the model's robustness and generalization under domain variability.

All volumes were resampled to a unified $128 \times 128 \times 128$ resolution for the experiments.

Key Insights and Ideas

  • Flow > Diffusion for Sparse Tasks: The deterministic nature of flow matching appears more robust than the stochastic denoising of diffusion when the "anchor points" (the vessels) are few and far between.
  • The Importance of Edges: Simply providing a mask isn't enough. By explicitly weighting the loss at the vessel-tissue interface, the model learns the spatial relationship between the vasculature and the surrounding anatomy much more effectively.
  • Multi-Scale is Mandatory: Without the auxiliary coarse-scale supervision, the model often produces local textures that look realistic but lack global anatomical logic.

Key Results

Vessel2Brain consistently outperformed state-of-the-art baselines, including 3D Pix2Pix (GAN), 3D Medical VAE, and Med-DDPM (Diffusion).

  • Quantitative: On the CereVessMRA dataset, Vessel2Brain achieved an SSIM of 65.36% and a PSNR of 26.84, significantly higher than the GAN-based Pix2Pix (SSIM 57.69%).
  • Qualitative: Visual inspections show that while other models produce "blurry" or anatomically disconnected structures, Vessel2Brain generates clear, sharp boundaries and maintains high structural consistency with the input vessel masks.
Fig. 2. Qualitative comparison on the CereVessMRA dataset. From left to right: vascu- lar mask, ground-truth brain MRI, 3D Pix2Pix, LDM, 3D VAE, Med-DDPM, and the proposed Vessel2Brain. Results are visualized on representative sagittal, coronal, and axial slices. The highlighted regions and corresponding zoomed-in views illustrate fine- grained anatomical details. Compared with baseline methods, Vessel2Brain generates more anatomically consistent brain structures while better preserving vascular-aligned tissue appearance under sparse vessel conditioning.

Limitations and Future Work

While impressive, the authors note several areas for growth:

  • One-to-Many Mapping: A single vessel mask could technically correspond to multiple valid brain appearances. Future work aims to incorporate uncertainty-aware sampling to better reflect this inherent ambiguity.
  • Multi-Contrast Generation: The current model focuses on a single MRI modality. Expanding this to generate T1, T2, and FLAIR images simultaneously from the same mask is a logical next step.
  • Scaling: The researchers plan to explore scaling strategies to improve generalization even further when paired training data is extremely limited.

Summary updated

Abstract

Generating realistic 3D brain MRI volumes from sparse anatomical priors is a highly ill-posed yet clinically valuable problem, with important applications in data augmentation, simulation, and downstream neuroimaging analysis. In this work, we investigate the challenging task of synthesising full 3D brain MRI volumes conditioned solely on vascular label maps. While vascular structures provide strong anatomical and topological cues, they occupy only a small fraction of the brain volume and offer no direct information about surrounding tissue appearance. To address this challenge, we propose Vessel2Brain, an edge-aware multi-scale conditional flow matching framework that explicitly incorporates vascular structure information into the generative process. Our approach formulates vessel-guided brain image synthesis as a conditional continuous flow, enabling stable training and flexible generation while preserving structural correspondence. A multi-scale flow matching strategy is introduced to jointly model global anatomical coherence and fine-grained local details, with auxiliary velocity predictions providing additional supervision across spatial resolutions. Furthermore, we design an edge-aware objective that emphasises vascular boundaries during training, encouraging anatomically faithful tissue synthesis around vessel structures. We evaluate Vessel2Brain on two public 3D brain MRI datasets with paired vascular annotations. Extensive quantitative and qualitative experiments demonstrate that our method consistently outperforms state-of-the-art generative models in terms of reconstruction accuracy, structural consistency, and distributional realism.
BibTeX
@misc{chen2026vessel,
  author = {Zhi Chen and Tianyang Zhang and Theodoros N. Arvanitis and Le Zhang},
  title = {{Vessel2Brain: Edge-Aware Multi-Scale Flow Matching for 3D Brain Image Generation from Vessel Masks}},
  year = {2026},
  eprint = {2611.17919},
  archivePrefix = {arXiv},
  url = {https://arxiv.org/abs/2611.17919},
}
Figure