Introduction
Early diagnosis of Alzheimer’s Disease (AD) is a critical clinical challenge. Currently, clinicians integrate structural neuroimaging, such as Magnetic Resonance Imaging (MRI), with cognitive assessments like the Mini-Mental State Examination (MMSE) and patient history from Electronic Health Records (EHR). However, automated systems often struggle to integrate these disparate data types effectively. Existing models frequently face "task interference," where optimizing for disease classification degrades the accuracy of cognitive score regression. Furthermore, processing high-resolution 3D MRI volumes within standard Vision Transformer (ViT) architectures is computationally prohibitive.
Methodology: The FLIQA-AD Architecture
To address these challenges, the authors introduce FLIQA-AD (Fusion Language Image Question Answering). This framework reframes AD diagnosis and MMSE prediction as a unified visual question-answering (VQA) task.
The architecture comprises four primary components:
- 3D Adapter: This module processes 3D MRI patches, projecting them into an embedding space that preserves spatial relationships without the need for 2D slicing or excessive flattening. It efficiently reduces feature dimensionality from 1,408 to 768, making the model computationally feasible.
- Bio-ClinicalBERT: EHR data, including age, gender, and education, are encoded using this domain-specific model, which was pre-trained on extensive medical corpora like PubMed and MIMIC-III.
- ADFormer (The Fusion Core): This module utilizes self-attention and cross-attention mechanisms to allow textual EHR "prompts" to guide the interpretation of visual MRI features. This ensures the model focuses on structural brain changes most relevant to the specific patient's demographic profile.
- LLM Decoder (FLAN-T5): Instead of simple linear layers, a Large Language Model reasons over the fused multimodal features to generate diagnostic labels or continuous MMSE scores as natural language outputs.
Datasets and Evaluation
The model was rigorously evaluated on the ADNI (Alzheimer’s Disease Neuroimaging Initiative) dataset, which includes over 8,300 images across three categories: Alzheimer’s (AD), Mild Cognitive Impairment (MCI), and Normal Control (NC). To test the model's robustness and "zero-shot" generalization capabilities, it was also evaluated on the OASIS-2 dataset without any prior fine-tuning on that specific data, providing a rigorous test of its performance across different imaging protocols.
Key Results
FLIQA-AD demonstrates state-of-the-art performance across both classification and regression tasks. On the ADNI dataset, it achieved a 97% overall accuracy for disease identification and a 0.98 AUC. For MMSE score prediction, the model reached a Coefficient of Determination ($R^2$) of 0.90 and a Pearson Correlation of 0.95, indicating an extremely tight fit between predicted and actual cognitive scores.
The model's zero-shot performance on the OASIS-2 dataset (81% accuracy for AD vs. NC) significantly outperformed other multimodal models like MedBLIP, which achieved only 20% in the same setting. As shown in the embedding visualization, the ADFormer output features form distinct clusters for AD, MCI, and NC patients, indicating that the model has learned a highly separable and clinically relevant representation space.
Conclusion and Future Work
FLIQA-AD represents a significant step forward in multimodal medical AI. By treating diagnosis as a reasoning task and leveraging the contextual power of EHR data through the ADFormer, the model overcomes the traditional trade-offs between classification and regression. While the model shows strong generalization, future work will focus on expanding its ability to handle more complex, open-ended clinical queries and further enhancing its robustness against domain shifts in diverse global datasets.
Summary updated