MedAlign: A Synergistic Framework of Multimodal Preference Optimization and Federated Meta-Cognitive Reasoning

Siyong Chen, Jinbo Wen, Jiawen Kang, Tenghui Huang, Xumin Huang, Yuanjia Su, Hudan Pan, Zishao Zhong, Dusit Niyato, Shengli Xie, Dong In KimarXiv 2025 · DOI · OpenReview

TL;DR

MedAlign tackles the critical challenges of deploying Large Vision-Language Models (LVLMs) in healthcare by introducing a novel framework that ensures visually grounded answers, optimizes reasoning efficiency, and enables multi-institutional collaboration. It achieves state-of-the-art performance in Medical Visual Question Answering (Med-VQA) by combining multimodal preference optimization, a retrieval-aware expert routing system, and a federated adaptive reasoning mechanism.

What Problem Does This Paper Address?

Deploying powerful LVLMs in clinical settings for tasks like Medical Visual Question Answering (Med-VQA) faces significant hurdles. These include:

  1. Hallucinations: LVLMs can generate answers that sound plausible but are not supported by the visual evidence in medical images, leading to potentially dangerous inaccuracies.
  2. Inefficient Reasoning: Standard models use a fixed depth for reasoning (e.g., Chain-of-Thought), which can be overkill for simple questions and insufficient for complex ones, wasting computational resources.
  3. Multi-institutional Collaboration: Training and deploying these models across different hospitals or institutions is challenging due to data privacy concerns and the need for standardized, reliable performance.

MedAlign aims to bridge this gap by creating a more trustworthy, efficient, and collaborative LVLM for medical applications.

Methods

MedAlign introduces a synergistic framework with three core components:

  1. Multimodal Direct Preference Optimization (mDPO): This is an extension of the Direct Preference Optimization (DPO) technique. Unlike standard DPO which primarily focuses on text, mDPO explicitly incorporates visual grounding. It uses a cross-modal preference loss to penalize answers that are visually incongruent with the provided medical image, directly addressing the hallucination problem. An anchor-based reward regularization is also used to stabilize training.
  2. Retrieval-Aware Mixture-of-Experts (RA-MoE): This architecture addresses the need for specialized and context-aware reasoning. Instead of relying solely on text-based routing or simple embedding similarity, RA-MoE uses multimodal retrieval scores to dynamically select the most relevant "expert" LVLM for a given query. This expert is then augmented with retrieved textual context, making the retrieval process an active part of the reasoning guidance rather than a passive addition.
  3. Federated Governance for Adaptive Reasoning: This component tackles reasoning inefficiency and multi-institutional collaboration. It employs a meta-cognitive uncertainty estimator that allows each selected expert to autonomously determine the optimal depth of its Chain-of-Thought (CoT) reasoning. This adaptive approach avoids unnecessary computation. Furthermore, it extends federation principles from training to inference, enabling multiple sites to collaborate on reasoning while maintaining privacy. A structured aggregation mechanism is used to synthesize results from different sites, ensuring consensus and resolving conflicts.

Datasets

The paper evaluates MedAlign on three widely used Med-VQA benchmarks:

  • MIMIC-CXR: A large dataset of chest radiographs with associated free-text reports.
  • IU-Xray: A dataset of chest X-rays and their corresponding radiology reports.
  • Harvard-FairVLMed: A dataset designed for multimodal reasoning in medical imaging.

The datasets were partitioned into training, validation, and testing sets (80%/10%/10%).

Key Insights and Ideas

  • Visual Grounding is Paramount: The paper strongly emphasizes that for medical applications, linguistic plausibility is insufficient; answers must be grounded in visual evidence. Their mDPO objective directly targets this.
  • Retrieval as Active Guidance: Moving beyond simple retrieval-augmented generation, MedAlign uses retrieval scores to route queries to specialized experts, making the retrieval process an integral part of the decision-making.
  • Adaptive Reasoning for Efficiency: The meta-cognitive uncertainty estimator is a clever way to dynamically control reasoning depth, leading to significant computational savings without sacrificing accuracy.
  • Federation for Collaborative Inference: Extending federated learning principles to inference is a novel approach for multi-institutional collaboration, addressing privacy concerns while enabling shared expertise.
  • Synergistic Components: The results show that each component of MedAlign contributes significantly, and their combination leads to performance gains that are more than the sum of their parts.

Key Results

  • State-of-the-Art Performance: MedAlign achieves new SOTA performance on all three Med-VQA benchmarks, outperforming strong retrieval-augmented baselines by up to 11.85% in F1-score.
  • Reduced Reasoning Length: The adaptive reasoning mechanism significantly reduces computational overhead, decreasing the average reasoning length by 51.60% compared to fixed-depth CoT approaches.
  • Robustness: MedAlign demonstrates superior robustness to both textual and visual noise, maintaining over 85% of its original F1-score even under 40% noise levels, unlike foundational models that degrade significantly.
  • Component Contributions: Ablation studies show that removing any single component leads to a noticeable performance drop, highlighting the effectiveness of mDPO, RA-MoE, and the federated governance mechanism.

Limitations and Future Work

  • Limitations:
    • While MedAlign shows improved robustness, it still experiences performance degradation with increasing noise levels, though less so than baselines.
  • The paper mentions that MedAlign has a slightly higher inference latency than non-retrieval baselines due to its components, though it remains competitive with powerful hybrid methods.
  • Future Work:
    • The authors plan to extend MedAlign beyond VQA to a broader range of clinical tasks.
  • They aim to enable collaborative inference among multiple experts, further advancing AI systems for real-world clinical services.

Summary updated

Abstract

Recently, large models have shown significant potential for smart healthcare. However, the deployment of Large Vision-Language Models (LVLMs) for clinical services is currently hindered by three critical challenges: a tendency to hallucinate answers not grounded in visual evidence, the inefficiency of fixed-depth reasoning, and the difficulty of multi-institutional collaboration. To address these challenges, in this paper, we develop MedAlign, a novel framework to ensure visually accurate LVLM responses for Medical Visual Question Answering (Med-VQA). Specifically, we first propose a multimodal Direct Preference Optimization (mDPO) objective to explicitly align preference learning with visual context. We then design a Retrieval-Aware Mixture-of-Experts (RA-MoE) architecture that utilizes image and text similarity to route queries to a specialized and context-augmented LVLM (i.e., an expert), thereby mitigating hallucinations in LVLMs. To achieve adaptive reasoning and facilitate multi-institutional collaboration, we propose a federated governance mechanism, where the selected expert, fine-tuned on clinical datasets based on mDPO, locally performs iterative Chain-of-Thought (CoT) reasoning via the local meta-cognitive uncertainty estimator. Extensive experiments on three representative Med-VQA datasets demonstrate that MedAlign achieves state-of-the-art performance, outperforming strong retrieval-augmented baselines by up to $11.85\%$ in F1-score, and simultaneously reducing the average reasoning length by $51.60\%$ compared with fixed-depth CoT approaches.
BibTeX
@misc{chen2025medalign,
  author = {Siyong Chen and Jinbo Wen and Jiawen Kang and Tenghui Huang and Xumin Huang and Yuanjia Su and Hudan Pan and Zishao Zhong and Dusit Niyato and Shengli Xie and Dong In Kim},
  title = {{MedAlign: A Synergistic Framework of Multimodal Preference Optimization and Federated Meta-Cognitive Reasoning}},
  year = {2025},
  eprint = {2510.21093},
  archivePrefix = {arXiv},
  url = {https://arxiv.org/abs/2510.21093},
}
Figure