Towards accurate and robust cross-ratio based gaze trackers through learning from simulation

Jia-Bin Huang, Qin Cai, Zicheng Liu, Narendra Ahuja, Zhengyou ZhangETRA 2014 · DOI

TL;DR

This paper introduces an adaptive homography mapping technique that significantly improves the accuracy and robustness of cross-ratio (CR) based gaze trackers. By learning to compensate for both spatially-varying gaze errors and head pose variations from simulation data, their method achieves state-of-the-art performance without requiring extensive subject-specific calibration.

What Problem Does This Paper Address?

Remote gaze tracking, which estimates where a person is looking using a camera, is crucial for applications like natural human-computer interaction and understanding attention. Cross-ratio (CR) based methods are attractive because they don't require complex hardware calibration and can handle some head movement. However, their accuracy degrades significantly as the user moves away from the initial calibration position, especially with changes in head depth. Existing methods often address accuracy and robustness separately, or rely on assumptions that limit their applicability (e.g., weak perspective cameras). This paper aims to unify these improvements into a single, learning-based framework that is both accurate at the calibration point and robust to head movements, all while minimizing the need for tedious subject calibration.

Methods

The core of this work is an adaptive homography mapping. This builds upon existing homography-based methods that correct biases in CR gaze estimation. The key innovation is making this homography adaptive, meaning it changes based on the user's current head position and gaze direction.

Here's a breakdown:

  • Base Method: They start with homography-based bias correction, which maps predicted gaze points from a basic CR method to ground truth targets. This correction is typically computed during a subject-specific calibration.
  • Adaptive Homography: The proposed method introduces a new homography, $H_A$, that modifies the bias-correcting homography ($H_{NL}$). This adaptive homography is learned to compensate for:
  • Spatially-varying errors: These arise from assumptions about the eye's geometry and the relationship between pupil and corneal reflections. The paper uses the pupil center's position in a normalized image space ($x_g$) as a predictor for these errors.
  • Head movement errors: These occur when the user's head position deviates from the calibration point. The paper uses geometric transformations of the glint patterns (reflections of infrared lights on the eye) to capture relative head movements ($x_m$).
  • Learning Framework: The adaptive homography $H_A$ is modeled as a quadratic regression function of the predictor variables ($x_m$ and $x_g$). This function is trained offline using simulated data. This allows for a rich set of head movements and eye parameters to be explored without collecting extensive real-world data for each user.
  • Training: The training process involves minimizing both algebraic and reprojection errors. The algebraic error minimization provides an initial estimate for the regression coefficients, which is then refined using non-linear least squares optimization.

Datasets

The paper primarily uses simulated data for training and evaluation. They generate synthetic eye images and ground truth gaze data by simulating:

  • A screen and camera setup with specified parameters (resolution, focal length, IR light positions).
  • A virtual eye model with typical eye parameters (corneal radius, pupil-corneal distance, angular deviations), with variations applied to test sensitivity.
  • A range of head positions (5x5x5 grid) and screen target grids for calibration and testing.
  • Random sampling of 50 virtual subjects to account for inter-subject variability.

They also evaluate their method on real data collected from a physical setup.

Key Insights and Ideas

  • Unified Compensation: The most significant insight is that spatially-varying gaze errors and head movement errors can be effectively compensated simultaneously within a single adaptive homography framework. This avoids the need for separate correction mechanisms.
  • Learning from Simulation: Training the adaptive homography using simulated data is a crucial practical advantage. It drastically reduces the time and effort required for subject-specific calibration, making the system more deployable.
  • Predictors for Adaptation: Using both the geometric transformation of glint patterns ($x_m$) and the pupil center's normalized position ($x_g$) as predictor variables for the adaptive homography is effective. This captures both global head motion and local gaze-dependent errors.
  • Quadratic Regression Sufficiency: The paper demonstrates that a quadratic regression model for the adaptive homography is sufficient to capture the complex variations in the optimal homography across different head positions and gaze directions, outperforming linear models.
  • Robustness to Calibration Position: The method's performance is shown to be insensitive to the initial calibration position, further enhancing its practical utility.

Key Results

  • Accuracy Improvement: The proposed adaptive homography method consistently outperforms existing CR-based methods (CR-DV, CR-HOM, CR-DD) in terms of gaze prediction accuracy, especially under head movements.
  • Robustness to Head Movement: The method shows remarkable robustness to head movements in X, Y, and Z directions. For example, when a subject moves 250mm towards the screen, their method maintains an error of less than 2°, while others exceed 5°. For movements parallel to the screen, errors are less than 0.3°.
  • Reduced Calibration Time: By learning from simulation, the need for extensive subject-specific calibration is significantly reduced.
  • Sensitivity Analysis: The method is shown to be relatively stable across variations in eye parameters, and its performance is influenced by sensor resolution and noise levels, as expected.

Limitations and Future Work

  • Camera Resolution: The paper notes that at very low camera resolutions or high noise levels, the input to the adaptive homography model can become unreliable, leading to degraded performance. This suggests that sufficient image quality is still a prerequisite.
  • Real-world Complexity: While validated on a physical setup, the paper acknowledges that real-world scenarios can introduce further complexities (e.g., varying lighting, occlusions) that might require additional handling.
  • Future Work: The authors suggest that incorporating more complex models for eye parameters or head pose, and exploring different learning architectures, could further enhance performance. They also mention the potential for online adaptation or learning in real-time.

Summary updated

BibTeX
@inproceedings{huang2014towards,
  author = {Jia-Bin Huang and Qin Cai and Zicheng Liu and Narendra Ahuja and Zhengyou Zhang},
  title = {{Towards accurate and robust cross-ratio based gaze trackers through learning from simulation}},
  booktitle = {ETRA},
  year = {2014},
  doi = {10.1145/2578153.2578162},
}
Figure