Appearance-based Eye Control System by Manifold Learning

Ke Liang, Youssef Chahir, Michèle Molina, Charles Tijus, François Jouen2014 · DOI

TL;DR

This paper introduces an appearance-based eye control system that uses a novel subregion Center-Symmetric Local Binary Pattern (CS-LBP) descriptor and manifold learning (specifically Laplacian Eigenmaps and spectral clustering) for efficient and robust eye movement recognition. The system achieves real-time performance with minimal calibration, making it suitable for HCI applications, especially for users with disabilities.

What Problem Does This Paper Address?

Eye control systems are crucial for enabling individuals with physical disabilities to interact with computers and communicate. Traditional gaze tracking methods often rely on feature-based approaches that require precise pupil and iris detection, infrared (IR) light, and extensive camera calibration. While appearance-based methods offer advantages by directly mapping eye images to gaze coordinates, they typically demand a high number of calibration points for accuracy. This paper addresses the need for a more efficient, less intrusive, and robust eye control system that requires minimal calibration and can adapt to variations in illumination and head movement.

Methods

The core of the proposed system lies in its novel approach to eye appearance description and manifold learning:

  • Appearance Descriptor: Instead of traditional feature extraction, the system uses a subregion Center-Symmetric Local Binary Pattern (CS-LBP) descriptor. The eye image is divided into subregions, and for each subregion, a CS-LBP histogram is computed. CS-LBP compares center-symmetric pixel pairs, making it robust to illumination changes. These histograms are concatenated to form a low-dimensional eye appearance feature vector.
  • Manifold Learning:
    • Spectral Clustering: Used during the calibration phase to group similar eye movement patterns from unlabelled data. This helps build an on-line model of the subject's eye movements by identifying distinct clusters of manifolds.
  • Laplacian Eigenmaps: Applied to learn the low-dimensional manifold structure of eye movements. This technique preserves proximity relations in the data, mapping nearby input patterns to nearby outputs. It's used to analyze the similarities between training samples and new inputs, enabling the inference of eye movements.

The system combines these techniques for both calibration (building a subject-specific model) and prediction (recognizing new eye movements).

Datasets

The paper does not explicitly mention or introduce a new benchmark dataset. Instead, it uses eye samples from five subjects for experimentation. The data collection involves subjects following visual patterns on a screen, with variations in illumination and head movement.

Key Insights and Ideas

  • CS-LBP Robustness: The CS-LBP descriptor proves to be significantly more robust to illumination changes compared to traditional subregion descriptors. This is a key advantage for real-world applications where lighting conditions can vary.
  • Efficient Calibration: The use of spectral clustering allows for an efficient on-line calibration process. By identifying clusters of eye movement manifolds, the system can build a predictive model with a limited number of training samples.
  • Manifold Structure for Recognition: The paper highlights how the learned manifold structure can be used to infer eye movements. Differences in manifold structure can even be used to detect events like eye blinks.
  • Minimal Calibration Requirement: The system demonstrates the ability to achieve reasonable accuracy with as few as 12 training samples, significantly reducing the calibration time and effort compared to other appearance-based methods.

Key Results

  • Illumination Invariance: Experiments show that the CS-LBP descriptor maintains the translation of eye movement structure under illumination changes, while a standard subregion descriptor is significantly disturbed.
  • Head Movement Tolerance: The system demonstrates a degree of robustness to slight head movements, preserving the eye movement structure, although the scale of the structure changes.
  • Real-time Potential: The computation time for Laplacian Eigenmaps with 12 training samples is approximately 41 ms, indicating the potential for real-time application.
  • Recognition Capabilities: The system can recognize four gaze directions (up-right, up-left, down-right, down-left) and also detect closed-eye movements and eye blinks.

Limitations and Future Work

  • Limited Dataset: The experiments were conducted with a small number of subjects and did not involve a large, standardized dataset, which could limit the generalizability of the findings.
  • Threshold Sensitivity: The performance of the system relies on empirically chosen thresholds (e.g., Tblinks and Tcorners), which might need fine-tuning for different users or conditions.
  • Future Work: The authors suggest that further research could explore more sophisticated manifold learning techniques and investigate the system's performance with a wider range of users and more complex HCI tasks. The paper also implicitly suggests that extending the system to handle more diverse eye movements and more complex interaction scenarios would be valuable.

Summary updated

Abstract

International audience
BibTeX
@misc{liang2014appearance,
  author = {Ke Liang and Youssef Chahir and Michèle Molina and Charles Tijus and François Jouen},
  title = {{Appearance-based Eye Control System by Manifold Learning}},
  year = {2014},
  doi = {10.5220/0004682601480155},
}

Loading related content...

Figure