Abstract
CodeBrain is a novel framework for unified MRI imputation, capable of generating missing brain MRI modalities (e.g., T1, T2, FLAIR) from any available subset of scans. By framing image synthesis as a discrete code prediction task using Finite Scalar Quantization (FSQ), CodeBrain captures fine-grained patient details more effectively than previous methods. The framework employs a two-stage approach: disentangled reconstruction to extract modality-agnostic anatomy and modality-specific codes, followed by a prior encoder to predict missing codes from incomplete data. Evaluated on IXI and BraTS 2023 datasets, CodeBrain achieves state-of-the-art performance in both quantitative metrics and downstream clinical utility.
Introduction
Magnetic Resonance Imaging (MRI) is essential in neurology, yet clinical datasets are frequently incomplete. A full suite of scans—including T1-weighted, T2-weighted, FLAIR, and contrast-enhanced T1Gd—is often unavailable due to cost, time constraints, or patient risks. This missing data hampers both clinical diagnosis and the performance of automated AI tools.
Existing imputation methods often suffer from two primary limitations: rigidity and complexity. Many are designed for specific "one-to-one" tasks (e.g., T1 to T2 translation) and fail to generalize. Conversely, "unified" models that attempt any-to-any translation often struggle to preserve instance-specific anatomical details or rely on overly complex, modality-specific modules. CodeBrain addresses these challenges by reframing imputation as a discrete "code prediction" task.
Methodology
CodeBrain utilizes a "divide-and-conquer" strategy implemented in two distinct stages.
Stage I: Disentangled Reconstruction
The model first learns to decompose an MRI into modality-agnostic features (representing the underlying brain anatomy) and modality-specific codes (capturing the style and intensity characteristics of a specific scan for a specific patient). Unlike traditional Vector Quantization (VQ) which requires learned codebooks, CodeBrain employs Finite Scalar Quantization (FSQ). FSQ maps image features to a finite set of integers, creating a discrete, compact "DNA" for each scan.
Stage II: Full-Modality Code Prediction
With the reconstruction mechanism established, the second stage trains a "prior encoder" to look at available partial scans and predict the missing codes for target modalities. A key innovation is the Grading Loss, an ordinal loss function that ensures predicted codes are numerically close to the ground truth, thereby preventing visual artifacts.
Experimental Evaluation
The framework was evaluated on two major benchmarks: the IXI Dataset (healthy subjects) and BraTS 2023 (brain tumor patients). BraTS is particularly challenging due to the irregular shapes and intensities of tumors.
Key Results
CodeBrain outperformed four state-of-the-art models, including transformer-based (MMT) and diffusion-based (M2DN) approaches.
- Quantitative Performance: On the IXI dataset, CodeBrain achieved a 0.89 dB improvement in Peak Signal-to-Noise Ratio (PSNR) over the previous best method.
- Structural Fidelity: Higher Structural Similarity Index (SSIM) scores indicate that imputed scans accurately preserve texture and anatomy.
- Clinical Utility: In downstream brain tumor segmentation tasks, scans generated by CodeBrain yielded Dice scores nearly identical to those achieved using ground-truth scans.
Discussion and Conclusion
CodeBrain demonstrates that treating MRI imputation as code prediction simplifies the learning objective while maintaining high anatomical consistency. By extracting shared features across modalities, the model ensures that imputed scans are structurally "locked" to the source data, reducing the risk of misaligning critical features like tumors.
Despite these advances, predicting contrast-enhanced (T1Gd) scans remains challenging due to biological factors not fully captured in non-contrast scans. Future work may involve integrating MRI physics to further ground the generative process in reality and addressing the inherent risks of "hallucination" in clinical AI deployment.
Summary updated