The iBUG Eye Segmentation Dataset

Bingnan Luo, Jie Shen, Yujiang Wang, Maja Pantic

TL;DR

This paper introduces the iBUG Eye Segmentation Dataset, the first dataset specifically designed for low-resolution eye segmentation, crucial for real-world applications like video conferencing. They demonstrate that deep learning models significantly outperform traditional methods on this challenging task, particularly for non-frontal faces.

What Problem Does This Paper Address?

Accurate eye segmentation, the process of identifying the iris and sclera in an image, is a fundamental step in many computer vision tasks, including biometrics and human-computer interaction. However, existing datasets primarily focus on high-resolution images, which are not representative of typical consumer-grade cameras used in everyday scenarios like video calls. These low-resolution images present significant challenges due to blurry boundaries, variations in pose, and lighting conditions. The paper aims to bridge this gap by creating a new dataset and evaluating existing and novel methods for eye segmentation in these challenging low-resolution conditions.

Methods

The authors compare two main categories of methods for eye segmentation:

  • Deformable Model-Based Methods: These include established techniques like Active Appearance Models (AAM), Ensemble Regression Trees (ERT), and Supervised Descent Methods (SDM). These methods rely on learning shape and texture models from training data and then fitting them to new images. They are sensitive to initialization, occlusion, and pose variations.
  • Deep Semantic Segmentation Methods: The paper focuses on DeepLab, a state-of-the-art approach that utilizes atrous convolutional neural networks (ACNN) combined with Conditional Random Fields (CRF). ACNN allows for a larger receptive field without increasing computational cost, enabling better capture of contextual information. The CRF is used as a post-processing step to refine segmentation boundaries.

The researchers also developed a pipeline for initializing these models, starting with face detection, then using ridge regression to predict eye bounding boxes, and finally employing mean shapes for initialization.

Datasets

The core contribution of this paper is the introduction of the iBUG Eye Segmentation Dataset. This dataset was created by manually annotating images from existing datasets: HELEN, 300VW, CVL, and Columbia Gaze.

Key characteristics of the iBUG dataset:

  • Low Resolution Focus: Primarily contains images captured under conditions similar to consumer-grade cameras.
  • Pose Variations: Includes both frontal and non-frontal faces.
  • Ground Truth: Provides both 30-point landmarks and pixel-level segmentation masks for the iris and sclera.
  • Challenging Conditions: Contains a small number of samples with bad illumination and glasses.

Key Insights and Ideas

  • The Need for Low-Resolution Data: The paper strongly argues that existing high-resolution datasets are insufficient for real-world HCI applications, highlighting the critical need for datasets like iBUG.
  • Deep Learning Dominance: Deep learning models, particularly DeepLab variants, demonstrate superior performance and robustness compared to traditional deformable models, especially when dealing with challenging poses.
  • Robustness to Pose: Deep models show a milder accuracy reduction on non-frontal faces, suggesting they are better equipped to handle the complexities of real-world head movements.
  • Iris and Sclera Segmentation: The dataset enables evaluation of both holistic eye segmentation and more specific tasks like iris-only or sclera-only segmentation, revealing different model strengths.

Key Results

The paper evaluates models based on Intersection over Union (IOU) accuracy and Receiver Operating Characteristic (ROC) curves.

  • Overall Performance: Deep segmentation methods (VGG-16+CRF and ResNet101+CRF) consistently outperformed deformable models (AAM, ERT, SDM) across holistic, iris, and sclera segmentation tasks.
  • Deep Model Comparison: ResNet101 generally performed better than VGG-16, attributed to its larger architecture for capturing finer eye features.
  • Sclera Segmentation Challenge: ResNet101's performance on sclera-only segmentation was relatively lower, potentially due to the dataset size not being sufficient for such a large model, leading to overfitting.
  • Robustness: Figure 7 in the appendix clearly shows that deep models maintain higher accuracy on profile faces compared to deformable models, indicating better robustness to pose variations.

Limitations and Future Work

  • Dataset Size for Large Models: The authors acknowledge that the dataset might not be large enough for training very deep models like ResNet101 from scratch, especially for specific tasks like sclera segmentation, leading to potential overfitting.
  • Shape Prior Integration: While deep models show promise, the paper suggests that future research could further improve segmentation performance by incorporating shape priors and post-processing shape models, which were not fully explored in this baseline study.
  • State-of-the-Art: The paper uses widely-used models as baselines, not necessarily the absolute state-of-the-art, to demonstrate the dataset's effectiveness. Future work could explore more advanced deep learning architectures.

Summary updated

Abstract

This paper presents the first dataset for eye segmentation in low resolution images. Although eye segmentation has long been a vital preprocessing step in biometric applications, this work is the first to focus on low resolutions image that can be expected from a consumer-grade camera under conventional human-computer interaction and / or video-chat scenarios. Existing eye datasets have multiple limitations, including: (a) datasets only contain high resolution images; (b) datasets did not include enough pose variations; (c) a utility landmark ground truth did not be provided; (d) high accurate pixel-level ground truths had not be given. Our dataset meets all the above conditions and requirements for different segmentation methods. Besides, a baseline experiment has been performed on our dataset to evaluate the performances of landmark models (Active Appearance Model, Ensemble Regression Tree and Supervised Descent Method) and deep semantic segmentation models (Atrous convolutional neural network with conditional random field). Since the novelty of our dataset is to segment the iris and the sclera areas, we evaluate above models on sclera and iris only respectively in order to indicate the feasibility on eye-partial segmentation tasks. In conclusion, based on our dataset, deep segmentation methods performed better in terms of IOU-based ROC curves and it showed potential abilities on low-resolution eye segmentation task.
BibTeX
@inproceedings{luo2018ibug,
  author = {Bingnan Luo and Jie Shen and Yujiang Wang and Maja Pantic},
  title = {{The iBUG Eye Segmentation Dataset}},
  booktitle = {ICCSW},
  year = {2018},
  doi = {10.4230/oasics.iccsw.2018.7},
}
Figure