Introduction
In safety-critical domains like oncology, integrating diverse data sources—such as mRNA expression, DNA methylation, and miRNA—is essential for accurate diagnosis. However, multimodal data often contains noise or irrelevant information that varies across individual samples. Traditional "black-box" fusion techniques typically assume uniform importance across all modalities, leading to suboptimal performance and a lack of transparency. This paper introduces Multimodal Dynamics, a framework designed to enhance trustworthiness by dynamically assessing data informativeness at both the feature and modality levels.
Methodology: Dynamical Fusion
The core innovation lies in a two-stage "Dynamical Fusion" strategy that filters noise and weights modalities based on their sample-specific reliability.
1. Feature-level Dynamics
To handle high-dimensional noise, the framework employs a sparse gating network for each modality. This network generates a sample-specific mask that selectively activates informative features while suppressing noise. By applying $\ell_1$ regularization, the model identifies the most relevant biomarkers for each specific patient, facilitating inherent explainability.
2. Modality-level Dynamics
Recognizing that different modalities offer varying levels of certainty for different samples, the authors utilize True-Class-Probability (TCP) as a confidence metric. Since the true class is unknown during inference, a dedicated confidence regression network is trained to estimate the TCP for each modality.
3. Integration
The final joint representation is formed by weighting each modality's features by its estimated confidence before concatenation and classification. This ensures that the model prioritizes the most "certain" data sources for every decision.
Experimental Evaluation
The framework was validated using four multiomics datasets from The Cancer Genome Atlas (TCGA): BRCA (Breast invasive carcinoma), LGG (Brain glioma), KIPAN (Kidney cancer), and ROSMAP (Alzheimer’s Disease). Each dataset comprises three modalities: mRNA, DNA methylation, and miRNA.
Results and Discussion
The Multimodal Dynamics approach consistently outperformed state-of-the-art (SOTA) fusion methods and single-modality baselines.
Performance Gains
On the BRCA dataset, the model achieved an accuracy of 87.7%, surpassing the previous SOTA (TMC) by 3.5%. Similar trends were observed across other metrics and datasets, demonstrating the robustness of dynamical fusion.
Explainability and Stability
Beyond accuracy, the model provides transparency. Qualitative analysis revealed that the features prioritized by the gating network often align with known biological biomarkers (e.g., KLK8, PTX3). Furthermore, the filtering of uninformative data resulted in lower prediction variance compared to standard concatenation methods.
Conclusion
Multimodal Dynamics addresses the critical need for trustworthy AI in medicine by moving away from static fusion. By dynamically evaluating informativeness at multiple scales, the framework achieves superior performance while offering insights into the underlying biological signals. Future work will focus on optimizing computational efficiency for scenarios involving a larger number of modalities and exploring robustness against noisy labels.
Summary updated