Abstract
Magnetic resonance imaging (MRI) offers multiple sequences that provide complementary views of brain anatomy and pathology. However, real-world datasets often exhibit variability in sequence availability due to clinical and logistical constraints. This variability complicates radiological interpretation and limits the generalizability of machine learning models that depend on consistent multimodal input. In this work, we propose an anatomy-guided and modality-agnostic framework for assessing disease-related abnormalities in brain MRI, leveraging structural context to enhance robustness across diverse input configurations. We introduce a novel augmentation strategy, Region ModalMix, which integrates anatomical priors during training to improve model performance when some modalities are absent or variable. We conducted extensive experiments on brain tumor segmentation using the Multimodal Brain Tumor Segmentation Challenge (BraTS) 2020 dataset (n=369). The results demonstrate that our proposed framework outperforms state-of-the-art methods on various missing modality conditions, especially by an average 9.68 mm reduction in 95th percentile Hausdorff Distance and a 1.36% improvement in Dice Similarity Coefficient over baseline models with only one available modality. Our method is model-agnostic, training-compatible, and broadly applicable to multi-modal neuroimaging pipelines, enabling more reliable abnormality detection in settings with heterogeneous data availability.
Introduction
In clinical practice, a complete set of MRI sequences—typically T1, T2, T1-contrast enhanced (T1ce), and FLAIR—is ideal for comprehensive diagnosis. However, constraints such as patient discomfort, emergency time limits, or high costs frequently lead to "modality-incomplete" datasets. Standard deep learning models are often "brittle," showing significant performance degradation when faced with missing inputs. While synthesis or complex fusion layers have been proposed, they often struggle with clinical variability. This research introduces a modality-agnostic approach that leverages anatomical priors to maintain high diagnostic accuracy regardless of the specific subset of MRI sequences available.
Methods
The primary innovation of this work is Region ModalMix (RMM), an anatomy-guided data augmentation strategy. Unlike traditional methods like "CutMix," which use random cuboid patches, RMM utilizes biological structure to guide the mixing process.
The RMM process involves:
- Anatomical Parcellation: The brain is divided into 101 distinct anatomical regions using SynthSeg+.
- Region-Based Mixing: During training, anatomical regions are randomly selected, and information is swapped across different MRI modalities for the same subject. For instance, thalamic features from a T2 scan might be mixed into a FLAIR scan's representation.
- Structural Coherence: By mixing within predefined anatomical boundaries, the augmented images remain biologically plausible, forcing the model to learn modality-invariant features.
The framework is model-agnostic and was evaluated using various architectures, including transformer-based models (mmFormer, M2FTrans) and convolutional networks (RFNet).
Experimental Setup
The framework was validated using the BraTS 2020 dataset, comprising 369 subjects (293 high-grade and 76 low-grade gliomas). Each subject provided four standard MRI sequences. The task involved segmenting three tumor subregions: the necrotic core (NCR/NET), edema (ED), and enhancing tumor (ET). To evaluate robustness, scenarios with 1, 2, or 3 missing sequences were simulated during testing.
Results
RMM consistently outperformed state-of-the-art methods across 15 different modality combinations. Key findings include:
- Boundary Precision: In single-modality settings, RMM achieved an average reduction of 9.68 mm in the 95th percentile Hausdorff Distance (HD95), indicating significantly improved boundary accuracy.
- Overlap Accuracy: There was a 1.36% improvement in the Dice Similarity Coefficient over baseline models when only one modality was available.
- Generalization: RMM also improved performance when all modalities were present, suggesting it acts as a powerful regularizer that enhances overall model generalization.
- Backbone Synergy: The most significant gains were observed when RMM was paired with the M2FTrans backbone, particularly for challenging regions like the Enhancing Tumor (ET) and Tumor Core (TC).
Discussion and Limitations
By treating brain anatomy as a constant "container" for varying modality "textures," RMM enables models to identify abnormalities more reliably. However, the method's effectiveness depends on the accuracy of the initial anatomical parcellation, which may be challenging in pediatric cases or highly distorted brain structures. Future work will explore learnable augmentations, application to specialized sequences like DWI/SWI, and validation on real-world clinical datasets with naturally occurring missing data.
Summary updated