Artificial intelligence to automate assessment of ocular and periocular measurements

Khizar Rana, Mark Beecher, Carmelo Caltabiano, Carmelo Macri, Yang Zhao, Johan Verjans, Dinesh SelvaEuropean 2024 · DOI

TL;DR

This paper introduces a deep learning model that automates the measurement of ocular and periocular structures, achieving accuracy comparable to human experts. This advancement promises to enhance the objectivity and efficiency of clinical assessments and pave the way for remote patient monitoring.

What Problem Does This Paper Address?

Accurate measurement of periocular structures, such as eyelid position, is crucial for diagnosing and monitoring various ophthalmological conditions, including thyroid eye disease, tumors, and trauma. Current manual measurement methods, like marginal reflex distance (MRD) 1 and 2, are subjective, operator-dependent, and can be challenging for certain patient groups (e.g., children, cognitively impaired adults). Furthermore, the increasing adoption of telehealth makes these manual, in-person measurements increasingly impractical. This paper aims to bridge this gap by developing an automated, AI-driven system for these measurements.

Methods

The researchers developed a deep learning model for facial landmark detection. They utilized the HRNet-v2 architecture, a well-established backbone network known for its ability to learn high-resolution representations, as their core model. The model was trained to predict ten specific periocular landmarks on facial images. Once these landmarks were detected, standard geometric calculations using the OpenCV library were employed to derive various periocular measurements, including marginal reflex distance (MRD) 1 and 2, palpebral fissure height (PFH), inner and outer intercanthal distances (IICD, OICD), interpupillary distance (IPD), and horizontal palpebral aperture (HPA). The model was trained using a mean squared error (MSE) loss function to compare predicted heatmaps with ground-truth heatmaps derived from manual annotations. Data augmentation techniques, including rotation, scaling, and horizontal flips, were applied to improve the model's robustness.

Datasets

The study utilized a dataset of 958 eyes from 479 participants prospectively enrolled from the Royal Adelaide Hospital ophthalmology clinic. The dataset was randomly split into training (70%) and evaluation (30%) sets. The images were manually annotated with ten periocular landmarks using the Labelbox platform. No new benchmark dataset was introduced; rather, existing manual annotation practices were digitized and used for training.

Key Insights and Ideas

  • Automation for Objectivity: The core idea is that by automating these measurements with a deep learning model, the inherent subjectivity and operator variability of manual methods can be significantly reduced, leading to more objective and reproducible clinical data.
  • Deep Learning for Robustness: The choice of HRNet-v2 and the use of data augmentation highlight the power of deep learning to handle variations in real-world images (lighting, angles, facial expressions), which are common challenges for traditional computer vision methods.
  • Telehealth Enabler: The development of an automated system that can accurately assess periocular measurements from images has direct implications for remote patient assessment, making telehealth a more viable and effective option for ophthalmological care.
  • Pixel-to-Millimeter Calibration: The use of a standardized adhesive dot sticker with a known diameter placed on the forehead for pixel-to-millimeter conversion is a practical and robust approach to ensure accurate scaling of measurements, addressing a potential challenge in previous studies.

Key Results

The AI algorithm demonstrated close agreement with human measurements. The mean absolute errors ranged from a low of 0.22 mm for IPD to 0.88 mm for IICD. Intraclass correlation coefficients (ICCs) indicated excellent reliability (ICC > 0.90) for most measurements (MRD1, MRD2, PFH, OICD, IICD, IPD), with HPA showing good reliability (ICC 0.84). The landmark detection model itself was highly accurate, achieving a mean error rate of 0.51% and a failure rate of 0%. Bland-Altman plots confirmed good agreement between human and AI measurements, with most differences falling within acceptable clinical ranges.

Limitations and Future Work

The authors acknowledge several limitations:

  • Single-Center Study: The data was collected from a single center, which might limit the generalizability of findings to other Australian cohorts. A multicenter study would enhance generalizability.
  • In-Clinic Image Comparison: The AI system was compared with human measurements taken in the clinic on images. Future work should compare AI measurements with manual measurements taken in clinic to better assess performance under dynamic conditions (e.g., frontalis muscle action, subtle blinking).
  • Standardized Conditions: The photos were taken under standardized conditions. External validation on images from different cameras and settings is necessary before widespread use in Telehealth.

Despite these limitations, the study successfully demonstrates the potential of AI for objective and efficient periocular assessment, paving the way for streamlined remote patient care.

Summary updated

Abstract

The automated facial landmark detection network provided accurate and reliable periocular measurements. This may help increase the objectivity of periocular measurements in the clinic and may facilitate remote assessment of patients with tele-health.
BibTeX
@inproceedings{rana2024artificial,
  author = {Khizar Rana and Mark Beecher and Carmelo Caltabiano and Carmelo Macri and Yang Zhao and Johan Verjans and Dinesh Selva},
  title = {{Artificial intelligence to automate assessment of ocular and periocular measurements}},
  booktitle = {European},
  year = {2024},
  doi = {10.1177/11206721241249773},
}
Figure