Fundus2Globe: Generative AI-Driven 3D Digital Twins for Personalized Myopia Management

Danli Shi, Bowen Liu, Zhen Tian, Yue Wu, Jiancheng Yang, Ruoyu Chen, Bo Yang, Ou Xiao, Mingguang HearXiv 2025 · DOI · arXiv

Introduction

Myopia (nearsightedness) is a burgeoning global health crisis, projected to affect nearly half the world's population by 2050. In its severe form, pathological myopia, the eyeball undergoes significant structural deformation, stretching into an elongated shape that predisposes patients to vision-threatening complications like retinal detachment and posterior staphyloma. While Magnetic Resonance Imaging (MRI) is the current gold standard for assessing these 3D changes, its high cost and limited availability make it impractical for routine clinical use.

Fundus2Globe introduces a transformative generative AI framework that reconstructs high-fidelity 3D "digital twins" of the human eye using only standard 2D color fundus photographs (CFPs) and basic clinical metadata. By inferring complex 3D geometry from ubiquitous 2D data, this approach offers a scalable, low-cost solution for monitoring disease progression and personalizing myopia management.

Methodology

The Fundus2Globe pipeline integrates traditional geometric modeling with state-of-the-art generative AI through a multi-stage process:

  1. 3D Morphable Model (3DMM): To manage the complexity of 3D meshes, the researchers established a statistical "blueprint" of the eye. By applying Principal Component Analysis (PCA) to 3D eye shapes derived from MRI scans, they reduced high-dimensional geometry into a compact "ocular latent space."
  2. Latent Diffusion Model (LDM): The core engine is a diffusion model that operates within this latent space. It learns to "denoise" random noise into specific 3DMM parameters, effectively generating a 3D shape that matches the patient's unique ocular characteristics.
  3. Multimodal Conditioning: The generation process is guided by two primary inputs: a 2D Fundus Image processed via the EyeFound foundation model, and clinical metadata (Axial Length and Spherical Equivalent).
  4. Label Distribution Learning (LDL): Instead of treating clinical values as simple numbers, the authors utilized LDL to represent them as Gaussian distributions, helping the model capture the inherent relationships between similar clinical measurements.
The Fundus2Globe pipeline: (a) Pre-processing MRI to create real 3D meshes, (b) 3D Morphable Model for eye shape, and (c) the Latent Diffusion Model conditioned on fundus images and metadata.

Experimental Setup and Datasets

The model was developed and validated using two primary datasets:

  • Internal Development: The ZOC-BHVI Guangzhou High Myopia Cohort Study, featuring 197 eyes with paired 2D fundus photos and high-resolution 3D MRI scans.
  • External Validation: The PALM dataset, containing 1,200 fundus images, was used to evaluate the model's generalizability in identifying pathological myopia based on generated 3D geometry.

Key Results and Insights

The framework demonstrates remarkable precision and efficiency:

  • Submillimeter Accuracy: Fundus2Globe achieved a Mean Absolute Error (MAE) of 0.2048 mm in spatial deviations, effectively matching the structural fidelity of MRI.
  • Efficiency: While an MRI scan can take up to an hour and cost thousands of dollars, Fundus2Globe generates a 3D digital twin in under 3 minutes.
  • Clinical Utility: The model accurately captures "asphericity" (non-roundness), a critical biomarker for myopia severity. In external testing, the generated 3D shapes successfully distinguished between healthy and pathologically myopic eyes.
Overall performance comparison using Chamfer distance, showing the superiority of the proposed method (ours) and its variant with test-time augmentation (ours + tta).

A significant finding was the impact of metadata; incorporating simple clinical values like axial length reduced reconstruction errors by 18%, resolving ambiguities present in 2D images alone. Furthermore, the generative nature of the model allows for "counterfactual" simulations, enabling clinicians to visualize how a patient's eye might deform if their axial length were to increase.

Limitations and Future Directions

While highly effective at capturing global globe shape, the model currently lacks the microscopic resolution provided by Optical Coherence Tomography (OCT). Future iterations aim to integrate OCT data to enhance local anatomical detail. Additionally, expanding the training dataset to include a wider variety of ocular pathologies will further improve robustness. Future work may also incorporate biomechanical properties to move from static to dynamic digital twins.

Summary updated

BibTeX
@misc{shi2025fundus,
  author = {Danli Shi and Bowen Liu and Zhen Tian and Yue Wu and Jiancheng Yang and Ruoyu Chen and Bo Yang and Ou Xiao and Mingguang He},
  title = {{Fundus2Globe: Generative AI-Driven 3D Digital Twins for Personalized Myopia Management}},
  year = {2025},
  eprint = {2502.13182},
  archivePrefix = {arXiv},
  url = {https://arxiv.org/abs/2502.13182},
}
Figure