Appearance-based gaze tracking with spectral clustering and semi-supervised Gaussian process regression

Ke Liang, Youssef Chahir, Michèle Molina, Charles Tijus, François Jouen

Abstract

Appearance-based gaze tracking faces two primary challenges: prediction accuracy and the efficiency of the calibration process. This paper introduces a model designed for rapid calibration using a novel three-stage pipeline. By concatenating Center-Symmetric Local Binary Pattern (CS-LBP) descriptors from eye image subregions, the system forms a robust feature vector. Spectral clustering is then employed to extract supervision information from eye manifolds online. Finally, a sparse semi-supervised Gaussian Process Regression (GPR) estimates gaze coordinates. Experimental results show that a 5-point calibration yields an accuracy of 0.9°, significantly reducing run-time costs while maintaining high precision.

Introduction

Gaze tracking technology is traditionally divided into feature-based and appearance-based methods. Feature-based systems, while accurate, rely on expensive infrared hardware to track corneal reflections. Appearance-based methods offer a cost-effective alternative using standard webcams but are often hampered by "calibration exhaustion"—requiring extensive user input—and sensitivity to environmental lighting. This research addresses these hurdles by proposing a system that achieves high accuracy with minimal calibration points and robust performance under varying illumination.

Methodology

Feature Extraction via CS-LBP

To ensure robustness against lighting changes, the system moves beyond raw pixel intensities. The eye image is divided into 40 subregions, and a Center-Symmetric Local Binary Pattern (CS-LBP) descriptor is calculated for each. This method compares the gray levels of symmetric pixel pairs, focusing on local textures rather than absolute brightness.

The CS-LBP feature extraction process, showing subregion division and the local binary pattern calculation.

Manifold Learning and Spectral Clustering

The high-dimensional data of eye images can be represented as points on a low-dimensional manifold. Even as the eye moves through thousands of pixel configurations, the underlying geometry of the movement is constrained. The authors use Laplacian Eigenmaps to project these features into a 3D space, revealing a consistent topological structure across different subjects.

Visualization of eye manifolds in 3D space, demonstrating the underlying geometric structure of gaze data.

Sparse Semi-supervised Gaussian Process Regression

Mapping these manifold features to screen coordinates is achieved through Gaussian Process Regression (GPR). To maintain real-time performance, a "sparse" approach is used, selecting an "active set" of the most informative samples (both labeled calibration points and unlabeled frames). This allows the model to learn from natural eye movements, effectively "filling in the gaps" between sparse calibration points.

Results and Discussion

The proposed system demonstrates significant improvements in both efficiency and accuracy:

  • Accuracy: With only a 5-point calibration, the system achieved a mean error of 0.9°, outperforming other semi-supervised methods like $S^3GP$ (1.32°).
  • Efficiency: The sparse GPR prediction is highly optimized, requiring only 0.55ms per eye image.
  • Robustness: The use of CS-LBP descriptors allows the manifold structure to remain stable even under drastic shifts in indoor lighting, where raw intensity methods typically fail.

Conclusion

By combining texture-robust features with manifold learning and semi-supervised regression, this paper presents a viable path toward low-cost, high-accuracy gaze tracking. While future work is needed to address significant head movements and broader demographic testing, the 0.9° accuracy with a rapid 5-point calibration marks a substantial step toward user-friendly eye-tracking interfaces.

Summary updated

Abstract

Two of the challenges in appearance-based gaze tracking are: 1) prediction accuracy, 2) the efficiency of calibration process, which can be considered as the collection and analysis phase of labelled and unlabelled eye data. In this paper, we introduce an appearance-based gaze tracking model with a rapid calibration. First we propose to concatenate local eye appearance Center-Symmetric Local Binary Pattern(CS-LBP) descriptor for each subregion of eye image to form an eye appearance feature vector. The spectral clustering is then introduced to get the supervision information of eye manifolds on-line. Finally, taking advantage of eye manifold structure, a sparse semi-supervised Gaussian Process Regression(GPR) method is applied to estimate the subject's gaze coordinates. Experimental results demonstrate that our system with an efficient and accurate 5-points calibration not only can reduce the run-time cost but also can lead to a better accuracy result of 0.9°.
BibTeX
@misc{liang2013appearance,
  author = {Ke Liang and Youssef Chahir and Michèle Molina and Charles Tijus and François Jouen},
  title = {{Appearance-based gaze tracking with spectral clustering and semi-supervised Gaussian process regression}},
  year = {2013},
  doi = {10.1145/2509315.2509318},
}
Figure