This paper introduces SEEC (Standardized External-Eye Corpus), a massive new resource designed to push the boundaries of how AI understands the human eye. While most ophthalmic AI focuses on the internal structures of the eye (like the retina), this work focuses on the "external" eye—the eyelids, lashes, and surrounding skin—which can be photographed cheaply with any smartphone and contains vital clues about both ocular and systemic health.
TL;DR
The authors present SEEC, a 4.35-million-image dataset and benchmark for self-supervised learning of external-eye representations. Their findings reveal that domain-specific pretraining on eyes significantly outperforms general-purpose "foundation" models like DINOv2 on clinical tasks, even when using far less data.
What Problem Does This Paper Address?
External-eye photography is a low-cost, non-invasive way to screen for conditions like thyroid eye disease or ptosis (drooping eyelids), and even to find biomarkers for systemic diseases like diabetes. However, the field has lacked a large-scale, standardized dataset. Researchers have typically relied on small, task-specific datasets or general-purpose face datasets designed for identity recognition rather than clinical anatomy. This paper fills that gap by providing a massive, curated corpus and a multi-axis benchmark to evaluate how well AI models actually "understand" eye anatomy and disease.
Methods
The researchers developed a rigorous pipeline to curate the Standardized External-Eye Corpus (SEEC) from 14 public facial image sources.
The pipeline uses face detection and landmark alignment to extract precise, unilateral eye crops. They then trained ResNet-101 and ViT-B models using three different self-supervised learning (SSL) strategies:
- InfoNCE: A contrastive approach that learns by comparing similar and different images.
- VICReg: A method that prevents "representation collapse" by regularizing the variance and covariance of the embeddings.
- LeJEPA: A predictive approach that encourages models to predict missing parts of an image's representation.
The models were evaluated at two levels: the representation level (analyzing the mathematical geometry of the model's "brain space") and the functional level (testing on real-world tasks like landmark detection and disease classification).
Datasets
The study utilizes a hierarchy of datasets for training and evaluation:
- SEEC Master Corpus: 4.35 million unlabeled unilateral eye images.
- Pretraining Subsets: Fixed sets of 10k, 100k, and 1M images used to study how performance scales with data.
- Anatomical Probes (LM-Celeb, LM-CFD): Datasets with nine specific landmarks (like the iris center and eyelid corners) to test anatomical "knowledge."
- Clinical Sets (Clinic, Disease): Real-world clinical images used to test for "domain shift" and the ability to classify five distinct diseases, including thyroid eye disease and craniofacial syndromes.
Key Insights and Ideas
- Geometry $\neq$ Performance: One of the most striking findings is that "good" embedding geometry (how broadly a model uses its representation space) does not consistently predict how well it will perform on a clinical task. This suggests that standard metrics used to evaluate general AI models may be insufficient for medical applications.
- The Power of Domain Specificity: Models trained specifically on SEEC outperformed massive general-purpose models like DINOv2 (trained on 142M images) and MAE on anatomical tasks. This proves that for specialized medical fields, high-quality, relevant data is more important than sheer volume.
- No Universal Winner: No single SSL objective or architecture was optimal across all tasks. For example, LeJEPA produced the most "well-utilized" embedding spaces, but InfoNCE and VICReg often performed better on specific diagnostic tasks.
- Clinical Domain Shift: The models' internal representations changed significantly when moving from public face photos to "in-the-wild" clinical photos, highlighting the difficulty of deploying lab-trained models in real hospitals.
Key Results
- Disease Classification: Using a "frozen" model (where the AI's features are fixed and only a simple classifier is trained), the SEEC-pretrained ViT-B model achieved an impressive 0.91 Accuracy and F1-score on the five-class disease task.
- Anatomical Accuracy: In cross-dataset tests, the domain-specific ViT-B model achieved a Mean L2 error of 4.24, outperforming the much larger DINOv2 baseline (5.04).
- Scaling: While more data generally helped, the benefit of moving from 100k to 1M images was often smaller than the benefit gained by choosing the right training objective.
Limitations and Future Work
The authors acknowledge several limitations:
- Clinical Data Constraints: The labeled disease dataset is relatively small (633 images), and due to privacy and institutional restrictions, the clinical evaluation sets cannot be publicly released.
- Morphological Detail: The landmark probes use a simplified set of nine points, which may not capture the full complexity of eye shape and texture.
- Fine-tuning Sensitivity: Some models, particularly DINOv2, showed a tendency to "collapse" or lose performance when fine-tuned on small clinical datasets, suggesting they are highly sensitive to optimization settings.
Future work will focus on using more complex "multi-view" training techniques, longer training schedules, and larger model architectures to see if the performance gap over general-purpose models can be widened further.
Summary updated