QualitEye: Public and Privacy-preserving Gaze Data Quality Verification

Mayar Elfares, Pascal Reisert, Ralf Küsters, Andreas Bulling

Abstract

Gaze-based applications are increasingly advancing with the availability of large datasets but ensuring data quality presents a substantial challenge when collecting data at scale. It further requires different parties to collaborate, therefore, privacy concerns arise. We propose QualitEye--the first method for verifying image-based gaze data quality. QualitEye employs a new semantic representation of eye images that contains the information required for verification while excluding irrelevant information for better domain adaptation. QualitEye covers a public setting where parties can freely exchange data and a privacy-preserving setting where parties cannot reveal their raw data nor derive gaze features/labels of others with adapted private set intersection protocols. We evaluate QualitEye on the MPIIFaceGaze and GazeCapture datasets and achieve a high verification performance (with a small overhead in runtime for privacy-preserving versions). Hence, QualitEye paves the way for new gaze analysis methods at the intersection of machine learning, human-computer interaction, and cryptography.

Introduction

Gaze tracking is becoming ubiquitous, appearing in everything from VR headsets to smartphones. However, training robust gaze-estimation models requires massive, high-quality datasets. Currently, the community faces two major hurdles: Data Quality, where large-scale datasets are often plagued by "noisy" labels or inconsistent features; and Privacy, as eye images are highly sensitive biometric data. Privacy regulations and proprietary concerns often prevent researchers from sharing raw data to verify its quality against a trusted reference.

QualitEye is the first framework designed to verify the quality of image-based gaze datasets while strictly preserving user privacy. By combining deep learning for feature disentanglement with cryptographic private set intersection (PSI), it allows researchers to ensure that "similar eye features correspond to similar labels" across different datasets without ever exchanging raw, sensitive eye images.

Methodology

The QualitEye pipeline moves from raw pixels to a privacy-preserving "cryptographic handshake" through several stages.

Semantic Disentanglement

Instead of comparing raw pixels, QualitEye uses a Variational Auto-Encoder (VAE) to disentangle eye images into three distinct latent codes: Appearance, Gaze Direction, and Head Pose.

The QualitEye VAE architecture for disentangling eye images into semantic hashes and labels.

By intentionally ignoring the "Appearance" code (which contains identity-revealing features), the system focuses only on task-relevant information. This makes the method domain-agnostic and enhances privacy.

Locality-Sensitive Hashing (LSH)

To handle the inherent noise in gaze data, the authors use LSH. Unlike standard cryptographic hashes, LSH ensures that similar gaze/pose features result in the same hash value. This allows for "fuzzy" matching, which is essential for tolerating small systematic errors in data acquisition.

Privacy-Preserving Protocols

QualitEye implements several versions of Private Set Intersection (PSI) to compare these hashes and their associated labels between a Data Owner ($O$) and a Reference Party ($R$). These range from Diffie-Hellman based exchanges for small datasets to protocols using Oblivious Pseudorandom Functions (OPRF) and Cuckoo hashing for large-scale data.

Evaluation and Results

The system was evaluated on two major benchmarks: MPIIFaceGaze (213,200 images from 15 participants) and GazeCapture (~2.5 million frames from over 1,450 participants).

Distribution overlap between GazeCapture and MPIIFaceGaze, showing how QualitEye identifies matching regions across gaze direction and head pose.

QualitEye demonstrated high accuracy in distinguishing between "compliant" data (where features match labels) and "non-compliant" data. Using the Matthews Correlation Coefficient (MCC), the system achieved scores of 0.92 on MPIIFaceGaze and 0.94 on GazeCapture for gaze direction. Notably, even when comparing data across different datasets (MPIIFaceGaze vs. GazeCapture), the system maintained an MCC of 0.93, proving its cross-domain robustness. The privacy-preserving overhead is manageable; a cross-domain verification of thousands of samples took approximately 152 seconds on a standard GPU setup.

Discussion and Future Work

A key insight of QualitEye is using disentanglement as a "privacy filter," stripping away identity-specific features before hashing. Furthermore, the asymmetric verification setups allow data owners to verify against massive reference sets without needing to download them or requiring the reference party to be constantly online.

Current limitations include a dependency on the quality and distribution of the reference dataset. If the reference doesn't cover the same distribution of gaze angles as the owner's data, it will flag valid samples as mismatches. Future work could incorporate temporal information (scanpaths or video) to verify dynamic gaze behavior and move from a "semi-honest" to a "malicious" security model for stronger guarantees.

Summary updated

Abstract

Gaze-based applications are increasingly advancing with the availability of large datasets but ensuring data quality presents a substantial challenge when collecting data at scale. It further requires different parties to collaborate, therefore, privacy concerns arise. We propose QualitEye--the first method for verifying image-based gaze data quality. QualitEye employs a new semantic representation of eye images that contains the information required for verification while excluding irrelevant information for better domain adaptation. QualitEye covers a public setting where parties can freely exchange data and a privacy-preserving setting where parties cannot reveal their raw data nor derive gaze features/labels of others with adapted private set intersection protocols. We evaluate QualitEye on the MPIIFaceGaze and GazeCapture datasets and achieve a high verification performance (with a small overhead in runtime for privacy-preserving versions). Hence, QualitEye paves the way for new gaze analysis methods at the intersection of machine learning, human-computer interaction, and cryptography.
BibTeX
@misc{elfares2025qualiteye,
  author = {Mayar Elfares and Pascal Reisert and Ralf Küsters and Andreas Bulling},
  title = {{QualitEye: Public and Privacy-preserving Gaze Data Quality Verification}},
  year = {2025},
  eprint = {2506.05908},
  archivePrefix = {arXiv},
  url = {https://arxiv.org/abs/2506.05908},
}
Figure