openEyeTrack - A high speed multi-threaded eye tracker for head-fixed applications

Jorge Casas, Chandramouli ChandrasekaranJ. Open Source Softw 2019 · DOI · DBLP

Abstract

openEyeTrack is an open-source software solution designed for high-speed, video-based eye tracking in head-fixed research environments. By implementing a multi-threaded architecture, the system overcomes the performance limitations of traditional single-threaded trackers, achieving acquisition rates of up to 715 frames per second (fps) using affordable, off-the-shelf hardware. The software decouples image capture, pupil detection, display, and network communication into independent threads, ensuring lossless data processing and low latency for real-time behavioral control.

Introduction

In neuroscience and psychology, tracking eye movement is essential for understanding decision-making and cognition. However, researchers often face a difficult choice: buy expensive commercial "black box" systems that offer high performance but no transparency, or use open-source tools that are often too slow for high-speed applications. Most existing open-source trackers are single-threaded, meaning they capture, analyze, and display frames sequentially. If the pupil detection algorithm takes too long on one frame, the system lags or drops subsequent frames. This lack of temporal resolution makes it difficult to accurately detect rapid eye movements like saccades and microsaccades, which are critical for real-time behavioral experiments. openEyeTrack addresses these challenges by providing a high-performance, transparent, and cost-effective alternative.

Methods

The core innovation of openEyeTrack is its multi-threaded pipeline, which decouples the various stages of eye tracking into independent threads that communicate via queues. The software is built on a stack of open-source tools, including OpenCV for image processing, Qt5 for the interface, and the GigE-V framework for Linux-based camera communication.

System architecture of openEyeTrack showing the multi-threaded pipeline.

The pipeline consists of four primary thread types:

  1. Capture Thread: Grabs raw frames from a high-speed industrial camera (e.g., Teledyne DALSA Genie Nano) and stores them in a capture queue.
  2. Detection Threads: Multiple user-specified threads pop frames from the queue, convert them to OpenCV objects, and apply a blob detection algorithm to identify the pupil.
  3. Display Thread: Pulls processed frames to show the user a real-time visualization.
  4. Network Thread: Packages coordinates and pupil data into UDP packets for downstream applications or data logging.

By running multiple detection threads simultaneously, the software can match the high frame rate of the camera. Environmental optimizations, such as using an infrared (IR) light source and binary thresholding, further reduce the computational load on the blob detector.

Results

The openEyeTrack system demonstrates significant performance gains over traditional single-threaded solutions. Key results include:

  • Acquisition Speed: The system achieved frame acquisition rates of up to 715 fps.
  • Display Speed: Real-time display rates reached up to 145 fps.
  • Lossless Processing: The authors found that four detection threads were sufficient to maintain lossless processing at peak speeds, ensuring every captured frame is analyzed.
  • Hardware Accessibility: A high-performance system can be assembled for approximately $600–$700 in hardware costs, significantly lowering the barrier to entry for research labs.

Beyond positional tracking, the software also calculates pupil area, allowing researchers to study physiological arousal and emotional states alongside gaze data.

Discussion and Future Work

While openEyeTrack provides a high-speed solution for head-fixed applications, it currently has limitations. Because it only detects the pupil, any head movement is confounded with eye movement, restricting its use to head-fixed setups. To support head-free tracking, future versions will implement corneal reflection (glint) detection. Additionally, the software currently lacks analog output, which is often used to interface with legacy systems. Future work will also involve more rigorous validation in active research environments to identify further performance enhancements and provide additional metrics.

Summary updated

BibTeX
@article{casas2019openeyetrack,
  author = {Jorge Casas and Chandramouli Chandrasekaran and J. Paolo Casas},
  title = {{openEyeTrack - A high speed multi-threaded eye tracker for head-fixed applications}},
  journal = {J. Open Source Softw},
  year = {2019},
  doi = {10.21105/joss.01631},
}
Figure