Abstract
Developing high-performance artificial intelligence for ophthalmology typically requires massive amounts of expert-labeled data, which is both expensive and time-consuming to acquire. While foundation models pre-trained on unlabeled images offer a solution, existing approaches often rely on costly clinical reports for supervision or fail to account for the specific geometric constraints of retinal photography. This paper introduces PRETI (Patient-Aware Retinal Foundation Model), a novel framework that integrates easily accessible metadata—specifically age and gender—directly into the self-supervised learning process. By combining a learnable metadata embedding with a retina-specific adaptive masking strategy, PRETI captures both broad anatomical structures and fine-grained pathological details more effectively than previous models.
1. Introduction
In the field of medical imaging, foundation models have emerged as a powerful tool for reducing the dependency on labeled datasets. However, current retinal foundation models face two primary challenges. First, many rely on "report supervision," which requires matching images to text-based clinical reports. These reports are often unavailable or expensive to generate at scale. Second, standard self-supervised techniques like Masked Autoencoders (MAE) treat all image patches equally. In retinal fundus photography, the relevant clinical information is confined to a circular region, while the surrounding black background constitutes irrelevant noise that can degrade learning efficiency.
PRETI addresses these gaps by utilizing widely available patient metadata to guide representation learning and optimizing the model architecture specifically for the geometry of the human eye.
2. Methodology
PRETI introduces three key innovations designed to improve the quality of learned retinal representations:
2.1 Learnable Metadata Embedding (LME)
Rather than treating age and gender as static inputs, PRETI utilizes learnable tokens prepended to image patches. During the pre-training phase, the model is tasked with predicting the patient's age and gender via a specialized "Meta Loss." This objective forces the encoder to evolve its visual features to better represent biological variations associated with aging and sex, effectively using metadata as a biological prior.
2.2 Patient-Level Data Pairs
The model employs a Siamese architecture to process pairs of images from the same individual (e.g., the left and right eye). Because the two eyes of a single patient are biologically symmetrical, this approach encourages the model to ignore non-clinical noise—such as variations in camera brands or lighting conditions—and focus on the intrinsic, stable characteristics of the patient.
2.3 Retina-Aware Adaptive Masking (RAAM)
To address the structural inefficiency of standard MAE, PRETI implements RAAM, which features two distinct components:
- Region-Specific Masking: The model only masks patches within the circular retinal area, ignoring the uninformative black background.
- Cosine Decay Schedule: Training begins with an extremely high masking ratio (98.5%), forcing the model to reconstruct global structures from minimal fragments. As training progresses, the ratio gradually drops to 85%, allowing the model to refine its understanding of fine details like small blood vessels and lesions.
3. Experimental Setup
PRETI was pre-trained on a massive in-house dataset comprising 1,017,549 color fundus photographs (CFPs) from over 292,000 patients, including data from the UK Biobank.
The model's performance was evaluated across a diverse range of tasks:
- Internal Tasks: Detection of Diabetic Retinopathy (DR), Glaucoma, and Age-related Macular Degeneration (AMD), as well as the prediction of systemic biomarkers like Coronary Artery Calcium (CAC) and kidney function (eGFR).
- Public Benchmarks: Seven diverse datasets, including APTOS2019, MESSIDOR-2, IDRID, and PAPILA.
4. Results
PRETI consistently outperformed established foundation models, such as RETFound and UrFound, across nearly all evaluation metrics.
- Disease Prediction: On internal datasets, the PRETI (ViT-B variant) achieved an AUROC of 0.982 for DR and 0.777 for Glaucoma, significantly surpassing RETFound’s scores of 0.969 and 0.747, respectively.
- Biomarker Prediction: The model demonstrated a remarkable capacity for predicting systemic health markers, reaching an AUROC of 0.899 for CAC and 0.699 for eGFR.
- Generalization: On public benchmarks, PRETI matched or exceeded the performance of models trained with full clinical reports, suggesting that simple metadata can serve as a viable and efficient substitute for complex text data.
- Interpretability: Attention maps indicate that the LME-enabled model focuses more sharply on critical diagnostic regions, such as the optic cup and microaneurysms, compared to baseline models.
5. Discussion and Conclusion
The success of PRETI highlights the power of using metadata as a biological prior. By forcing the model to account for age and gender, it naturally learns to identify the macula, vessels, and optic disc with higher precision. Furthermore, the use of biological symmetry between eyes acts as a powerful regularizer, teaching the model to distinguish between clinical features and imaging artifacts.
Future work may involve expanding the variety of metadata used—such as smoking status or blood pressure—and extending this metadata-guided approach to 3D modalities like Optical Coherence Tomography (OCT). In conclusion, PRETI represents a significant step forward in creating accessible, high-performance foundation models for ophthalmology by leveraging the data already present in clinical workflows.
Summary updated