Pupil tracking method based on particle filtering in gaze tracking system

Jiannan Chi, Chuang Zhang, Yanjun Qin, Ying Li, Yang LiInternational Journal of the Physical Sciences 2011 · DOI

Abstract

In this paper, we propose a pupil tracking method based on particle filtering within an active infrared source gaze tracking system. We constructed a pupil target model combining gray-scale and shape characteristics to easily distinguish the target from the background. A pupil motion model was developed to predict location and shape features in subsequent frames. Furthermore, gray histogram similarity and geometrical shape similarity were combined to build a pupil observation model, improving the credibility of the particles. Experimental results in interactive graphical systems verify that our method is effective and feasible for real-time applications.

Introduction

Video-oculography (VOG) systems, which track gaze using infrared light and cameras, are essential for modern human-computer interaction. However, these systems often struggle with two primary issues: efficiency and robustness. Traditional tracking methods frequently re-locate the pupil across the entire image frame-by-frame, which is computationally expensive. Moreover, involuntary eye movements are non-linear, and events like blinking or external light interference can cause standard trackers to lose the target.

Existing solutions, such as Kalman filtering or Mean Shift, often fail because pupil motion is non-Gaussian, and these methods struggle to recapture the pupil once lost during a blink. This research introduces a robust tracking framework using Particle Filtering (PF) to handle these non-linearities and maintain a stable lock on the pupil.

Methodology

Initial Detection and Localization

Before continuous tracking begins, the system performs an initial localization of the eye region. This is achieved using the ADABOOST algorithm for face detection, followed by a "bright/dark pupil" differencing technique. By comparing adjacent frames captured under different infrared illumination states, the system can isolate the pupil from the complex background of the face.

The detailed algorithmic flowchart for pupil detection, including ADABOOST localization and image morphology filtering.

Particle Filter Framework

The core of the tracking system is a Particle Filter framework, which is uniquely suited for non-linear dynamical systems. The authors implemented several key innovations to enhance the PF:

  • Hybrid Observation Model: Instead of relying solely on pixel intensity, the system incorporates a shape-circularity feature ($R = 4\pi S / L^2$). Particle weights are calculated as a weighted sum of gray-scale similarity and geometric circularity, preventing the tracker from being distracted by noise that shares the pupil's color but not its shape.
  • Comprehensive State Vector: Each particle tracks 11 parameters, including $(x, y)$ coordinates, major and minor axes of the pupil ellipse, velocity, and target window dimensions.
  • Blink and Occlusion Handling: The system monitors particle weights. If all weights drop below a specific threshold—indicating a blink—the system pauses updates and retains the last known good state until the pupil reappears.
  • Adaptive Template Updating: To account for head orientation changes, the target template is updated dynamically based on the estimated state of previous frames, preventing tracking drift over time.

Experimental Results

The study utilized a custom dataset of 1,051 consecutive image frames collected from four different users under standard indoor lighting conditions. The proposed method was compared against standard color-histogram PFs and Kalman/Mean Shift hybrids.

Performance Metrics

  • Accuracy: The proposed method achieved a 99.14% average success rate. Notably, it maintained 99.22% accuracy during blinking states, where competing methods often failed.
  • Speed: The average execution time was 20ms per frame, meeting the requirements for real-time interactive systems.
  • Stability: Error analysis showed that the mean square error remained low and stable over 1,000 frames, while the errors for Kalman-based methods grew significantly after approximately 500 frames due to cumulative drift.
A close-up view of the pupil boundary fitting and center calculation result.

Conclusion and Future Work

The integration of geometric constraints and robust blink handling within a particle filter framework provides a highly effective solution for gaze tracking. While the system performs excellently in indoor environments, future work will explore performance in high-glare outdoor settings and with users wearing glasses. Additionally, further optimization could reduce the computational overhead inherent in particle filtering for mobile or resource-constrained platforms.

Summary updated

Abstract

In this paper we proposed a pupil tracking method based on particle filtering in active infrared source gaze tracking system. We constructed a pupil target model with the combination of gray characteristic and shape characteristic so that the target and the background can be distinguished easily. Pupil motion model which can predict pupil location and shape feature in the next frame was built. Gray histogram similarity and geometrical shape similarity were combined to build pupil observation model, which can improve the credibility of particles. In this paper we also showed the experiment results. The applications in interactive graphical system had verified that our method was effective and feasible. Key words: Gaze tracking, eye characters, particle filtering, face location, pupil detection, Purkinje segment.
BibTeX
@article{chi2011pupil,
  author = {Jiannan Chi and Chuang Zhang and Yanjun Qin and Ying Li and Yang Li},
  title = {{Pupil tracking method based on particle filtering in gaze tracking system}},
  journal = {International Journal of the Physical Sciences},
  year = {2011},
  doi = {10.5897/ijps11.067},
}
Figure