TL;DR
This paper introduces a novel, highly efficient approach for real-time pupil segmentation using Haar-like features and statistical learning. It achieves impressive speed (1000 segmentations per second) while maintaining robust performance, making it ideal for resource-constrained applications like mobile eye tracking and high-speed eye trackers.
What Problem Does This Paper Address?
Accurate pupil detection and segmentation are crucial for a wide range of applications, from driver monitoring and virtual reality to medical diagnostics and human-computer interaction. However, existing methods often struggle with challenges like varying illumination conditions, reflections on glasses, and the need for real-time performance on devices with limited computational power. Many state-of-the-art approaches are computationally expensive, making them unsuitable for mobile or energy-saving eye tracking systems. This paper aims to bridge this gap by developing a method that is both fast and accurate, even under challenging real-world conditions.
Methods
The core of this approach lies in a combination of Haar-like features and statistical learning. Instead of traditional integral image computations for Haar features, the authors opt for downscaling images to compute pixel differences, significantly speeding up feature extraction. These Haar-like features are then used in conjunction with conditional pupil shape probabilities to improve robustness against noise and fluctuating light.
The training process is particularly efficient. It involves three passes:
- Creating the search area and initial ellipse set (EL): This establishes a conditional probability distribution of possible ellipses at each location.
- Reducing EL and creating feature extractors: Ellipses are represented by eight landmarks, and similar landmark configurations are grouped to reduce redundancy. Haar features are then extracted from downscaled images.
- Weighting features: Individual features are weighted based on their reliability and contribution to determining pupil shape. This statistical weighting helps in identifying ellipses even when they are not fully visible.
The final detection involves calculating landmark differences across all possible positions, finding the minimum deviation weighted by feature importance, and using the conditional probability distribution to select the most likely ellipse.
Datasets
The authors utilized a dataset of segmented pupils from a previous work [38]. This dataset consists of two parts:
p1_image.mp4: Images taken in a driving simulator, characterized by simpler conditions with fewer reflections and light fluctuations. This set, with over 500,000 frames, was used for training.p2_image.mp4: Images from real-world driving, presenting more challenging conditions with reflections and strong light fluctuations. This set, with over 350,000 frames, was used for evaluation.
Data augmentation techniques were applied, including adding random noise, reflections, contrast adjustments, random shifting, and zooming.
Key Insights and Ideas
- Haar Features Reimagined: By using downscaled images instead of integral images for Haar feature computation, the authors achieve a significant speedup without sacrificing too much accuracy. This is a clever optimization for real-time applications.
- Statistical Learning for Robustness: Combining Haar features with conditional pupil shape probabilities and feature weighting makes the system more resilient to noise and partial occlusions, a common problem in eye tracking.
- Efficient Training: The three-pass statistical learning approach allows for very fast training, making it practical for personalizing detectors to individual users.
- Generalization of Binary Comparisons: The approach extends the binary comparison idea from algorithms like BORE and CBF by using more sophisticated Haar-like features and statistical modeling.
Key Results
- Speed: The proposed method achieves an impressive speed of "1000 Pupil Segmentations in a Second," making it ideal for high-speed and real-time applications.
- Performance: While not always reaching the absolute highest accuracy of some deep learning methods (like TinyCNNs) in terms of segmentation quality, it offers a strong balance between speed and accuracy. For instance, it reaches nearly 90% at a pixel error of two.
- Robustness: The method demonstrates good coverage across the image space and is more robust to reflections than some other methods like BORE.
- Training Efficiency: The training time is significantly lower than that of deep learning approaches, with the proposed method requiring only 1 hour of training compared to 40-44 hours for TinyCNNs.
Limitations and Future Work
- Search Area Limitation: The current approach is limited to finding pupils within a predefined search area. While this can be extended, it impacts detection rate and runtime.
- Overfitting to Training Data: The statistical weighting can lead to overfitting if the training data is not diverse enough, causing features valid in those specific images to be over-weighted.
- Shape Recognition: The detector only recognizes shapes present in the training data. While this can be addressed with simulated data or data manipulation, it increases runtime.
The authors suggest that their approach could be used in a two-stage process: an initial offline segmentation using a deep neural network, followed by the proposed statistical learning method for online, personalized pupil detection. This would leverage the strengths of both approaches.
Summary updated