TL;DR
Evo-RAD introduces a self-evolving agentic retrieval framework designed to overcome the limitations of medical foundation models when diagnosing rare retinal diseases. By framing retrieval as a Markov Decision Process, the system dynamically purges misleading, visually similar "hub" cases and inserts pathologically consistent evidence. This approach significantly outperforms traditional static retrieval and parameter-efficient fine-tuning methods while requiring minimal trainable parameters.
What Problem Does This Paper Address?
Large-scale vision-language foundation models have achieved remarkable success in general medical screening. However, they frequently stumble when encountering rare diseases because these conditions are severely underrepresented in real-world training datasets.
To avoid expensive full-model retraining, researchers often turn to Retrieval-Augmented Diagnosis (RAD), which pulls similar reference cases from an external database to help guide the model's prediction. Unfortunately, standard RAD pipelines rely on static, one-shot nearest-neighbor retrieval. In high-dimensional feature spaces, this triggers the hubness problem: a small subset of common-disease images (hubs) repeatedly appear as the closest neighbors to completely unrelated queries. As a result, the system retrieves images that look visually similar but are semantically incorrect, biasing the final diagnosis toward common conditions and away from the true rare disease.
Methods
Evo-RAD addresses the rigidity of static retrieval by transforming evidence acquisition into an iterative, context-aware reasoning process modeled as a Markov Decision Process (MDP).
The framework operates through the following core components:
- Graph-Based State Representation: The retrieved reference set is modeled as a graph. Nodes represent the images (encoded with visual and statistical features). Edges represent semantic consistency, which is calculated using fine-grained clinical tags (e.g., "bone-spicule pigmentation") generated by expanding the raw disease labels using a Large Language Model (LLM).
- Agentic Actions: A Graph Convolutional Network (GCN) serves as the policy agent. At each step, it observes the graph topology and chooses one of three discrete actions:
DELETE: Purge a discordant hub distractor from the reference set.INSERT: Add a pathologically consistent sample from a pre-fetched candidate buffer.TERMINATE: Conclude the evolution process once the reference set is diagnostically coherent.- Relational Policy Learning via GRPO: The agent is optimized using Group Relative Policy Optimization (GRPO)—an RL algorithm that eliminates the need for a separate critic network by utilizing group statistics to establish a dynamic baseline. The agent is driven by a homogeneity-aware reward that penalizes diagnostic discordance and rewards label purity and semantic density.
- Evidence-Conditioned Prediction: Once the agent chooses to terminate, a simple majority vote over the remaining, purified reference set determines the final diagnosis.
Datasets
Because standard public fundus benchmarks are heavily saturated and frequently included in the pretraining of major foundation models, the authors evaluated Evo-RAD on the Retina Image Bank (RIB), a continuously updated clinical repository. They curated two subsets:
- Rare-20: Comprising 20 rare retinal diseases, with each class containing between 15 and 75 images.
- Retina-31: Created by adding 11 common retinal diseases (each with >200 images) to the Rare-20 cohort, simulating a highly imbalanced, realistic clinical environment.
Both datasets were evaluated using a standard 70/15/15 train/val/test split.
Key Insights and Ideas
- Mimicking Clinical Reasoning: Instead of relying on passive similarity matching, Evo-RAD mimics how human clinicians work—iteratively gathering consistent evidence while actively ruling out clinical mimics.
- The Power of Fine-Grained Semantics: Standard disease labels are often too coarse to distinguish rare conditions from common lookalikes. By leveraging an LLM to expand labels into detailed clinical tag embeddings, the framework builds a much more reliable semantic graph for the agent to navigate.
- Extreme Parameter Efficiency: Rather than adjusting the heavy weights of a foundation model, the framework achieves its gains entirely by optimizing the context fed into the prediction head. The GCN policy network requires only 116.4K trainable parameters.
Key Results
- Substantial Gains Over Foundation Models: On the Rare-20 benchmark, Evo-RAD achieved 46.28% accuracy, outperforming the zero-shot performance of the strong RetiZero foundation model by an absolute 21.04%.
- Outperforming PEFT and Static RAD: Evo-RAD outperformed state-of-the-art Parameter-Efficient Fine-Tuning (PEFT) methods (such as BiomedCoOp) and traditional static retrieval approaches by 3.56% on Rare-20 accuracy, proving the value of dynamic set refinement.
- Sensitivity to Initial Retrieval Size ($K$): Hyperparameter tuning revealed that performance peaks when the initial retrieval size is set to $K=8$. Smaller values fail to provide enough context for the agent to manipulate, while larger values introduce excessive noise and redundant references.
Limitations and Future Work
The primary limitation acknowledged in the paper is that Evo-RAD currently operates under a single-label assumption. In real-world clinical practice, patients frequently present with multiple overlapping retinal conditions (co-morbidities). The authors note that future work will focus on extending the agentic framework to multi-label settings, allowing the evidence evolution process to navigate complex, mixed-pathology scenarios.
Summary updated