Evo-RAD: Navigating Rare Retinal Disease Diagnosis via Self-Evolving Agentic Retrieval

Wangding Xia, Ye Du, Jiashi Lin, Meng Wang, Danli Shi, Shujun WangarXiv 2026 · arXiv

TL;DR

Evo-RAD introduces a self-evolving agentic retrieval framework designed to overcome the limitations of medical foundation models when diagnosing rare retinal diseases. By framing retrieval as a Markov Decision Process, the system dynamically purges misleading, visually similar "hub" cases and inserts pathologically consistent evidence. This approach significantly outperforms traditional static retrieval and parameter-efficient fine-tuning methods while requiring minimal trainable parameters.

What Problem Does This Paper Address?

Large-scale vision-language foundation models have achieved remarkable success in general medical screening. However, they frequently stumble when encountering rare diseases because these conditions are severely underrepresented in real-world training datasets.

To avoid expensive full-model retraining, researchers often turn to Retrieval-Augmented Diagnosis (RAD), which pulls similar reference cases from an external database to help guide the model's prediction. Unfortunately, standard RAD pipelines rely on static, one-shot nearest-neighbor retrieval. In high-dimensional feature spaces, this triggers the hubness problem: a small subset of common-disease images (hubs) repeatedly appear as the closest neighbors to completely unrelated queries. As a result, the system retrieves images that look visually similar but are semantically incorrect, biasing the final diagnosis toward common conditions and away from the true rare disease.

Methods

Evo-RAD addresses the rigidity of static retrieval by transforming evidence acquisition into an iterative, context-aware reasoning process modeled as a Markov Decision Process (MDP).

Fig. 1: Overview of Evo-RAD. Evo-RAD models retrieval as a Markov De- cision Process where a graph policy evolves the evidence set through discrete DELETE, INSERT, and TERMINATE operations.

The framework operates through the following core components:

  1. Graph-Based State Representation: The retrieved reference set is modeled as a graph. Nodes represent the images (encoded with visual and statistical features). Edges represent semantic consistency, which is calculated using fine-grained clinical tags (e.g., "bone-spicule pigmentation") generated by expanding the raw disease labels using a Large Language Model (LLM).
  2. Agentic Actions: A Graph Convolutional Network (GCN) serves as the policy agent. At each step, it observes the graph topology and chooses one of three discrete actions:
  3. DELETE: Purge a discordant hub distractor from the reference set.
  4. INSERT: Add a pathologically consistent sample from a pre-fetched candidate buffer.
  5. TERMINATE: Conclude the evolution process once the reference set is diagnostically coherent.
  6. Relational Policy Learning via GRPO: The agent is optimized using Group Relative Policy Optimization (GRPO)—an RL algorithm that eliminates the need for a separate critic network by utilizing group statistics to establish a dynamic baseline. The agent is driven by a homogeneity-aware reward that penalizes diagnostic discordance and rewards label purity and semantic density.
  7. Evidence-Conditioned Prediction: Once the agent chooses to terminate, a simple majority vote over the remaining, purified reference set determines the final diagnosis.

Datasets

Because standard public fundus benchmarks are heavily saturated and frequently included in the pretraining of major foundation models, the authors evaluated Evo-RAD on the Retina Image Bank (RIB), a continuously updated clinical repository. They curated two subsets:

  • Rare-20: Comprising 20 rare retinal diseases, with each class containing between 15 and 75 images.
  • Retina-31: Created by adding 11 common retinal diseases (each with >200 images) to the Rare-20 cohort, simulating a highly imbalanced, realistic clinical environment.

Both datasets were evaluated using a standard 70/15/15 train/val/test split.

Key Insights and Ideas

  • Mimicking Clinical Reasoning: Instead of relying on passive similarity matching, Evo-RAD mimics how human clinicians work—iteratively gathering consistent evidence while actively ruling out clinical mimics.
  • The Power of Fine-Grained Semantics: Standard disease labels are often too coarse to distinguish rare conditions from common lookalikes. By leveraging an LLM to expand labels into detailed clinical tag embeddings, the framework builds a much more reliable semantic graph for the agent to navigate.
  • Extreme Parameter Efficiency: Rather than adjusting the heavy weights of a foundation model, the framework achieves its gains entirely by optimizing the context fed into the prediction head. The GCN policy network requires only 116.4K trainable parameters.

Key Results

  • Substantial Gains Over Foundation Models: On the Rare-20 benchmark, Evo-RAD achieved 46.28% accuracy, outperforming the zero-shot performance of the strong RetiZero foundation model by an absolute 21.04%.
Table 1: Comparison with foundation models under zero-shot and linear probing settings. Performance is reported as mean±std for linear probing evaluation.
  • Outperforming PEFT and Static RAD: Evo-RAD outperformed state-of-the-art Parameter-Efficient Fine-Tuning (PEFT) methods (such as BiomedCoOp) and traditional static retrieval approaches by 3.56% on Rare-20 accuracy, proving the value of dynamic set refinement.
Table 2: Comparison with retrieval and PEFT methods (mean±std). RetiZero is the foundation model. Params is number of trainable parameters on Retina-31.
  • Sensitivity to Initial Retrieval Size ($K$): Hyperparameter tuning revealed that performance peaks when the initial retrieval size is set to $K=8$. Smaller values fail to provide enough context for the agent to manipulate, while larger values introduce excessive noise and redundant references.
Impact of initial retrieval size K

Limitations and Future Work

The primary limitation acknowledged in the paper is that Evo-RAD currently operates under a single-label assumption. In real-world clinical practice, patients frequently present with multiple overlapping retinal conditions (co-morbidities). The authors note that future work will focus on extending the agentic framework to multi-label settings, allowing the evidence evolution process to navigate complex, mixed-pathology scenarios.

Summary updated

Abstract

Large-scale pretrained foundation models have revolutionized general medical screening, but often falter on rare diseases because such conditions are underrepresented in real-world clinical datasets. While retrieval-augmented diagnosis attempts to mitigate this, conventional static methods frequently succumb to the hubness problem, retrieving visually similar but semantically incorrect common diseases. To address this, we propose Evo-RAD, a self-evolving agentic framework that transforms evidence acquisition into a dynamic decision-making task. We formulate retrieval as a Markov Decision Process (MDP) where a graphbased agent observes the reference set state and executes actions to purge discordant evidence (DELETE), acquire pathologically consistent samples (INSERT), or conclude the evolution (TERMINATE). Optimized via Group Relative Policy Optimization (GRPO) with a homogeneityaware reward, the agent learns to maximize the diagnostic homogeneity of the support reference set. Experiments on retinal disease benchmarks show that Evo-RAD substantially improves rare-disease diagnosis, outperforming retinal foundation models by +21.04%, while also surpassing retrieval-based and parameter-efficient fine-tuning methods by +3.56%. Code is available at https://github.com/SDH-Lab/Evo-RAD.
BibTeX
@misc{xia2026evo,
  author = {Wangding Xia and Ye Du and Jiashi Lin and Meng Wang and Danli Shi and Shujun Wang},
  title = {{Evo-RAD: Navigating Rare Retinal Disease Diagnosis via Self-Evolving Agentic Retrieval}},
  year = {2026},
  eprint = {2606.22955},
  archivePrefix = {arXiv},
  url = {https://arxiv.org/abs/2606.22955},
}
Figure