Introduction
Diffusion models have emerged as a powerful paradigm for image generation, yet their application in 3D medical imaging is often constrained by high computational costs and the scarcity of large-scale annotated datasets. In clinical settings, precise spatial control is paramount—for instance, synthetic images must strictly adhere to provided anatomical masks or partial scans. While 2D control mechanisms like ControlNet have been successful for natural images, they are not inherently optimized for the volumetric nature of medical data.
This paper introduces the Volumetric Conditioning Module (VCM), a lightweight, asymmetric plug-in designed to provide precise spatial control over pretrained 3D diffusion models. By freezing the base model and training only the VCM, researchers can achieve high-fidelity results even with extremely limited data.
Methodology
The VCM is designed as an architectural "sidecar" to a frozen 3D diffusion backbone (e.g., BrainLDM). This modular approach preserves the extensive anatomical knowledge of the pretrained model while introducing specific spatial guidance.
Key Innovations
- Asymmetric U-Net Architecture: VCM utilizes a deep encoder to capture complex 3D spatial cues and a shallower decoder to map these features into the base model's latent space. This asymmetry is optimized for parameter efficiency and 3D data compression.
- Feature Modulation: Instead of simple additive coupling, VCM predicts modulation parameters ($\gamma$ and $\beta$) to scale and shift the internal feature maps of the base model, allowing for more granular and effective guidance.
- Modality Latent Dropout: To handle diverse conditioning inputs such as masks and partial images, a dropout strategy is employed during training. This ensures the model remains robust even when certain inputs are missing during inference.
- Time-Conditioning: The module incorporates diffusion timestep embeddings, allowing it to adjust the strength of spatial guidance as the generation process progresses from noise to a detailed reconstruction.
Experimental Results
The VCM was evaluated using the ADNI database (604 T1-weighted brain MRI scans) across three training regimes: 10, 50, and 500 scans. The module was tested on tasks including semantic map-to-MRI synthesis, partial image completion, and axial super-resolution.
Data Efficiency and Alignment
VCM demonstrated exceptional performance in data-scarce environments. With only 10 training samples, it achieved a Dice score of 0.877 for ventricle-guided synthesis, significantly outperforming standard Latent Diffusion Models (LDMs) trained from scratch (0.656). In larger-scale tests (500 samples), VCM achieved a Hausdorff Distance (HD95) of 13.346, outperforming the T2I-Adapter (16.231) and matching the performance of the much heavier ControlNet (12.000) while being significantly more efficient.
Axial Super-Resolution
A key application of VCM is axial super-resolution, where thick-slice scans are enhanced to thin-slice quality. By treating this as a spatial conditioning problem, VCM achieved a PSNR of 27.518 for 8x super-resolution, vastly outperforming specialized methods like SynthSR (19.020). This success highlights VCM's ability to leverage the rich anatomical priors already present in the pretrained diffusion backbone.
Conclusion
The Volumetric Conditioning Module provides a robust and efficient framework for steering 3D medical diffusion models. Its ability to achieve high-fidelity results with as few as 10 scans makes it a valuable tool for medical research where data is limited. Future work will focus on extending VCM to other anatomical regions and evaluating its performance on pathological cases.
Summary updated