PupilTAN: A Few-Shot Adversarial Pupil Localizer

Nikolaos Poulopoulos, Emmanouil Z. Psarakis, Dimitrios I. KosmopoulosWorkshops @ CVPR 2021 · DOI

Introduction

Eye center localization is a fundamental component of human-computer interaction (HCI) and gaze tracking systems. Despite its importance, achieving high precision remains challenging due to variations in illumination, head pose, and occlusions from eyewear or hair. While modern deep learning models have significantly improved localization accuracy, they typically rely on large-scale, manually labeled datasets. Manual labeling is not only resource-intensive but also prone to pixel-level inaccuracies.

PupilTAN addresses these challenges by introducing a "few-shot unsupervised" framework. Instead of requiring thousands of labels, the system learns from the statistical distribution of eye centers, achieving state-of-the-art performance with minimal supervision.

Methodology

PupilTAN treats eye localization as an unsupervised image-to-heatmap translation task. The framework consists of several key stages:

1. Statistical Prior Estimation

The authors observed that eye centers within cropped regions of interest (ROIs) follow a predictable spatial distribution. By analyzing a small sample of just 128 images, they calculated the Maximum Likelihood Estimate (MLE) of eye center coordinates, determining that they follow a 2D Gaussian distribution.

The estimated 2D Gaussian PDF of eye center coordinates used as a statistical prior.

2. Adversarial Training Framework

The core of PupilTAN is a Generative Adversarial Network (GAN) architecture designed for heatmap translation:

  • The Translator (Generator): A three-stage encoder-decoder network that transforms a 64x64 grayscale eye image into a predicted heatmap. The goal is for the brightest pixel in the heatmap to correspond to the pupil center.
  • The Discriminator: This network is trained to distinguish between "fake" heatmaps produced by the Translator and "real" artificial heatmaps generated based on the previously calculated statistical prior.
The PupilTAN architecture showing the adversarial loop between the Translator and Discriminator.

Through this adversarial competition, the Translator learns to produce heatmaps that match the global statistical properties of real eye centers without ever seeing an individual ground-truth label for the training images.

Experimental Results

The model was evaluated on two primary benchmarks: the MUCT dataset (3,755 images) and the BioID dataset (1,521 images), both known for challenging lighting and low-resolution conditions.

  • Accuracy: PupilTAN achieved a "fine" localization accuracy ($e \le 0.05$) of 97.15% on MUCT and 96.86% on BioID. These results outperform several supervised state-of-the-art methods.
  • Efficiency: The model is highly optimized, with a lightweight version containing only 0.2M parameters. It achieves an inference speed of 16ms per image on standard hardware, making it suitable for real-time applications.
  • Unsupervised Advantage: Interestingly, the unsupervised PupilTAN outperformed its own supervised counterpart by 1.64%, suggesting that matching a statistical distribution provides a more robust training signal than noisy manual labels.

Conclusion

PupilTAN demonstrates that high-precision eye localization can be achieved with minimal labeled data by leveraging adversarial distribution matching. By shifting the focus from individual labels to statistical priors, the framework offers a robust, efficient, and scalable solution for real-time gaze tracking and HCI applications. Future work may involve integrating the system with more robust face detection pipelines to handle extreme head poses.

Summary updated

BibTeX
@inproceedings{poulopoulos2021pupiltan,
  author = {Nikolaos Poulopoulos and Emmanouil Z. Psarakis and Dimitrios I. Kosmopoulos},
  title = {{PupilTAN: A Few-Shot Adversarial Pupil Localizer}},
  booktitle = {Workshops @ CVPR},
  year = {2021},
  doi = {10.1109/cvprw53098.2021.00350},
}
Figure