I-ODA, Real-World Multi-modal Longitudinal Data for OphthalmicApplications

Nooshin Mojab, Vahid Noroozi, Abdullah Aleem, Manoj Prabhakar Nallabothula, Joseph Baker, Dimitri T. Azar, Mark Rosenblatt, R. V. Paul Chan, Darvin Yi, Philip S. Yu, Joelle A. Hallak

Introduction

The Illinois Ophthalmic Database Atlas (I-ODA) represents a significant advancement in medical imaging datasets, comprising over 3.6 million images from 33,876 patients collected over a 12-year period. By providing a diverse, multi-modal, and longitudinal perspective on real-world clinical data, I-ODA aims to bridge the "translational gap" that frequently prevents ophthalmic AI models from achieving clinical utility in hospital environments.

The Translational Gap in Ophthalmic AI

While AI has demonstrated human-level performance in detecting conditions such as diabetic retinopathy and glaucoma, these models are often trained on "clean" datasets from controlled clinical trials. In real-world clinical settings, performance often degrades due to variations in lighting, hardware brands, and complex patient comorbidities. Current ophthalmic AI research faces three primary hurdles: limited scale, a lack of longitudinal data for tracking disease progression, and "modality silos" where datasets focus on a single imaging type rather than the multi-modal approach used by clinicians.

The I-ODA Pipeline: Processing "Dark Data"

Transforming retrospective clinical records into a structured dataset requires a sophisticated engineering pipeline. The authors developed several key components to organize and anonymize the data:

  • Automated Modality Tagging: To label millions of raw files lacking explicit metadata, a ResNet-50 neural network was employed to extract image features. These features were compared against "prototype" images of known modalities using similarity-based classification, cross-referenced with device metadata.
  • Data Integration: The system merges three disparate sources: an internal image server, a SQL database of patient metadata, and hospital billing systems for diagnoses and surgical records.
  • Smart Anonymization: To ensure HIPAA compliance, K-means clustering was used to group images by layout, allowing for the creation of specific masking filters to redact patient identifiers from scan printouts.
The I-ODA automated modality tagging pipeline utilizes ResNet-50 features and device metadata to classify raw clinical images.

Dataset Characteristics

I-ODA stands apart from existing benchmarks like Messidor or IDRiD due to its scale and variety. It includes 12 distinct modalities, capturing the raw, unvarnished output of a major university eye infirmary over a decade.

The I-ODA dataset includes a wide variety of modalities, including B-Scans, OCT, Corneal Topography, and Fundus photography.

The dataset's longitudinal nature allows researchers to track the same eye over years, facilitating the development of predictive models for disease progression rather than just static detection. Key modalities include Fundus photography, Optical Coherence Tomography (OCT), B-Scans (ultrasound), Corneal Topography, and Humphrey Visual Fields (HVF).

Clinical Impact and Generalization

The variability inherent in real-world data—often seen as a weakness—is leveraged here as a strength to improve model robustness. In experimental evaluations, the authors demonstrated:

  • Glaucoma Detection: Pre-training on I-ODA for optic cup segmentation yielded a 2.6% improvement over standard baselines.
  • Generalization: Models trained on I-ODA's "messy" data showed a 16% relative improvement in performance when transferred to external standardized datasets, suggesting its utility as a "foundation" for ophthalmic AI.

Conclusion and Future Work

While I-ODA relies on retrospective billing codes for ground truth, its scale and diversity offer a blueprint for unlocking "dark data" in medical institutions. Future work will focus on expanding the databank and establishing I-ODA as a primary benchmark for validating algorithms before clinical deployment.

Summary updated

Abstract

Data from clinical real-world settings is characterized by variability in quality, machine-type, setting, and source. One of the primary goals of medical computer vision is to develop and validate artificial intelligence (AI) based algorithms on real-world data enabling clinical translations. However, despite the exponential growth in AI based applications in healthcare, specifically in ophthalmology, translations to clinical settings remain challenging. Limited access to adequate and diverse real-world data inhibits the development and validation of translatable algorithms. In this paper, we present a new multi-modal longitudinal ophthalmic imaging dataset, the Illinois Ophthalmic Database Atlas (I-ODA), with the goal of advancing state-of-the-art computer vision applications in ophthalmology, and improving upon the translatable capacity of AI based applications across different clinical settings. We present the infrastructure employed to collect, annotate, and anonymize images from multiple sources, demonstrating the complexity of real-world retrospective data and its limitations. I-ODA includes 12 imaging modalities with a total of 3,668,649 ophthalmic images of 33,876 individuals from the Department of Ophthalmology and Visual Sciences at the Illinois Eye and Ear Infirmary of the University of Illinois Chicago (UIC) over the course of 12 years.
BibTeX
@inproceedings{mojab2021i,
  author = {Nooshin Mojab and Vahid Noroozi and Abdullah Aleem and Manoj Prabhakar Nallabothula and Joseph Baker and Dimitri T. Azar and Mark Rosenblatt and R. V. Paul Chan and Darvin Yi and Philip S. Yu and Joelle A. Hallak},
  title = {{I-ODA, Real-World Multi-modal Longitudinal Data for OphthalmicApplications}},
  booktitle = {HEALTHINF},
  year = {2021},
  url = {https://openreview.net/forum?id=XsqHDC5Z2I},
}
Figure