Introduction
Medical Visual Question Answering (VQA) is a critical task where AI systems answer natural language questions about medical images. However, its development is hindered by the scarcity of labeled medical data. Traditional transfer learning from natural image datasets often fails due to the significant domain gap between everyday photos and specialized medical scans like X-rays or pathology slides. While meta-learning offers a path toward rapid adaptation, it typically requires extensive labeled meta-data. This paper introduces Multiple Meta-model Quantifying (MMQ), a framework designed to maximize the utility of existing medical datasets by leveraging unlabeled and noisy data through iterative refinement.
Methods
The MMQ framework operates through three integrated modules designed to extract robust features from limited data:
- Meta-training: Utilizing Model-Agnostic Meta-Learning (MAML), the system trains models to adapt quickly across various visual tasks, such as identifying different organs or imaging modalities.
- Data Refinement: This module implements a self-improvement loop. The model auto-annotates high-confidence unlabeled images and filters out "noisy" labeled images where its prediction uncertainty is high, effectively cleaning and expanding the training pool.
- Meta-quantifying: Rather than selecting a single optimal model, MMQ identifies a diverse set of $n$ meta-models. Selection is guided by a Fuse Score ($S_F$) that balances individual model accuracy with feature diversity (measured via cosine similarity), ensuring a robust "committee" perspective on image features.
The features from these quantified meta-models are concatenated and integrated into a standard VQA architecture using attention mechanisms (such as BAN or SAN) to generate final answers.
Datasets
The researchers evaluated MMQ on two primary benchmarks:
- VQA-RAD: A clinical dataset containing 315 images and 3,515 questions.
- PathVQA: A large-scale pathology dataset with 4,998 images and over 32,000 question-answer pairs. For this dataset, the authors developed a meta-annotation schema categorizing images into 31 classes based on organ types to facilitate the meta-learning process.
Key Results and Discussion
MMQ demonstrates significant performance gains over previous state-of-the-art methods like MAML and MEVF across both datasets. As shown in the results table, the framework is particularly effective on challenging open-ended questions.
- VQA-RAD: On "Open-ended" questions, MMQ achieved 53.7% accuracy, representing a 9.8% improvement over the previous best (MEVF at 43.9%).
- PathVQA: On "Free-form" questions, MMQ reached 13.4% accuracy, outperforming MEVF (8.1%) by 5.3%.
Ablation studies confirmed that performance consistently scales with the number of refinement loops (up to 5) and the number of meta-models used (optimally 3-4), validating the hypothesis that a diverse committee of models provides superior feature representation compared to a single "best" model.
Conclusion
The MMQ framework proves that internal data refinement and multi-model quantification can overcome the limitations of small medical datasets without requiring external data sources. While the iterative training process increases initial computational overhead, the resulting system achieves a new state-of-the-art in medical VQA. Future work may explore applying this quantifying approach to other medical AI domains, such as automated diagnosis and image segmentation, where data scarcity remains a persistent challenge.
Summary updated