TL;DR
This paper introduces a novel gaze estimation method that significantly improves robustness to external light sources. By analyzing the relationship between corneal specular reflections (SRs) and the pupil's movable area, the method effectively distinguishes genuine SRs from imposter SRs caused by ambient light, leading to more reliable gaze tracking in real-world environments.
What Problem Does This Paper Address?
Gaze tracking systems often rely on near-infrared (NIR) illumination and cameras to capture eye movements. However, external light sources, particularly those with NIR components (like sunlight or halogen lamps), can create spurious corneal specular reflections (SRs). These "imposter" SRs can be mistaken for the genuine SR generated by the system's illuminator, making it difficult to accurately determine the pupil's position and thus the user's gaze. This significantly degrades the performance of gaze tracking systems in uncontrolled environments.
Methods
The proposed method tackles the challenge of external light in several key ways:
- Bandpass Filter (BPF): Instead of a long-pass filter (LPF) commonly used in NIR systems, the authors employ a bandpass filter (BPF) in front of the camera. This BPF is tuned to the specific wavelength of the NIR illuminator, effectively blocking out a wider range of external light wavelengths, including those from halogen lamps and sunlight.
- External Light Region Removal: A component labeling and size filtering approach is used to identify and remove large, bright regions in the image that are likely caused by external light, especially when the light source is behind the user. This pre-processing step helps isolate the eye region.
- Robust Pupil Detection: A novel pupil detection method is introduced that utilizes a histogram of the eye region, smoothed by an average filter. This approach is more resilient to shadows cast by eyelids or eyelashes, which can mimic pupil darkness under static thresholding methods. The method identifies global and local maxima in the histogram to accurately determine the pupil's boundaries.
- Genuine SR Discrimination: The core innovation lies in discriminating the genuine SR from imposter SRs. This is achieved by analyzing the relationship between the detected corneal SRs and the pupil's movable area (PMA). The system defines a "genuine pupil movable area" (GPMA) and an "imposter pupil movable area" (IPMA) based on the relative positions of the SRs and the pupil's calibration points. By examining the overlap (or lack thereof) between these areas and the pupil's current position, the method can confidently identify the genuine SR. Four scenarios of relative positioning between the external light source and the NIR illuminator are considered.
Datasets
Not explicitly discussed in the paper. The experiments were conducted using images captured by a Logitech C600 webcam with a resolution of 1600x1200 pixels. The authors collected data from 10 individuals for various experimental conditions.
Key Insights and Ideas
- BPF is Crucial: The choice of a bandpass filter over a long-pass filter is a significant factor in reducing the impact of external light, as it selectively allows the NIR illuminator's wavelength while blocking others.
- Geometric Relationships Matter: The key to robust SR discrimination isn't just detecting SRs, but understanding their spatial relationship with the pupil's movement. The GPMA/IPMA concept elegantly captures this.
- External Light is a "Position Game": The effectiveness of the proposed method hinges on the assumption that the external light source and the system's illuminator have distinct spatial relationships relative to the user's gaze. This allows for the differentiation of their respective SRs.
- Pupil Movable Area as a Reference: The pupil's predictable movement within a defined area (derived from calibration) acts as a crucial anchor for validating SRs.
Key Results
- Reduced Gaze Detection Error: The proposed method significantly reduces gaze detection error compared to methods that do not account for external light. For instance, in one experiment, the error dropped from 9.52° (using LPF) to 0.56° (using BPF).
- Improved SR Detection Accuracy: The system achieved an average of 95.28% correctly labeled genuine SR images under halogen lamp illumination, and over 96% for users wearing glasses.
- Robustness to Various Conditions: The method demonstrated robustness across different external light positions (above, right, below, left of the monitor), sunlight conditions, and even with users wearing glasses.
- Fast Processing Speed: The overall processing time for the method was measured at approximately 53 ms per frame, enabling operation at around 19 frames per second.
- Outperforms Other Methods: The proposed eye ROI detection method achieved 100% accuracy with a processing time of 17 ms, outperforming AdaBoost (96.25% accuracy, 104.985 ms) and Camshift (99.5% accuracy, 45.8675 ms).
Limitations and Future Work
- Third Scenario Challenge: The authors acknowledge that in the "third scenario" (where the GPMA and IPMA have a common overlapping area), gaze detection errors can occur. This is particularly problematic when the pupil center falls within this overlap.
- Future Work: To address the third scenario, the authors plan to research methods that utilize tracking information from successive frames. They also aim to test the system in more diverse environments, including outdoors, mobile settings, and car interiors, to further enhance its robustness.
Summary updated