Eye trackers in virtual and augmented reality headsets stream where a person is looking up to 120 times per second. Applications use that stream for rendering, interfaces, and prediction. The same stream is also a fingerprint. A small embedding network can re-identify 98.1% of 102 people from brief gaze recordings (Learning User Embeddings from Human Gaze for Personalised Saliency Prediction), and gaze also reveals personality, sex, and neurological conditions. The obvious learned defense borrows from domain adaptation, the subfield that trains a feature extractor to fool a "domain discriminator" so that features stop carrying information about where the data came from. The proposed transfer treats each user as a domain and trains the sanitizer until a classifier can no longer tell users apart. Variants of this already exist. One adversarially strips appearance identity from eye images to improve gaze accuracy (Generalizing Eye Tracking With Bayesian Adversarial Learning). Another plays a privatizer-versus-classifier game on faces (Generative Adversarial Privacy). A third rewards an agent for fooling an ensemble of subject classifiers on scanpath images (Reinforcement Learning for the Privacy Preservation and Manipulation of Eye Tracking Data).
All three share a mismatch with the real threat. Their adversary is a closed-set classifier over the training users. The realistic attacker is different: it has never seen those users, it enrolls new people from a short sample, and it matches them by embedding distance, exactly as Strohm et al. do. A sanitizer can scramble the decision boundaries of a fixed N-way classifier and still leave behind a stable per-person geometry that a fresh metric-learning attacker recovers. This proposal trains the sanitizer against an open-set adversary instead. The adversary is a Siamese embedding network, trained to pull together two recordings of the same unseen person. The sanitizer itself is causal, meaning it only looks backward in time, so it adds no lag. Success is judged by attackers trained from scratch on held-out people after sanitization, and by whether the 0.12-second lead of eye over head movement that powers viewport prediction (Jin et al.) survives.
Core Claim
The claim is that matching the training adversary to the deployment threat model matters more than the choice of adversarial machinery. Concretely, a causal sanitizer trained against a metric-learning identity adversary on episodically sampled user subsets should reach lower re-identification by freshly trained attackers on held-out users than either a closed-set gradient-reversal sanitizer or tuned smoothing, at matched utility. This is a domain-generalization framing. Every user is a domain, and invariance must hold for domains never seen in training. It borrows the episodic idea of exposing modules to unfamiliar partners (Li et al.), but applies it to sampling which users the adversary sees in each step rather than to classifier pairings.
Research Program
The sanitizer $s_\theta$ is a small causal temporal convolutional network. It maps a raw gaze stream $x_{1:t}$ (gaze angles, optionally pupil size) to a sanitized stream $\tilde{x}_t$ with no look-ahead. Its output stays in coordinate space, so existing consumers need no changes. Training alternates between two players. The adversary $e_\phi$ embeds fixed-length windows of sanitized gaze and is trained with a triplet margin loss over users drawn from a fresh random subset at each episode, which mirrors Strohm et al.'s attack. The sanitizer then minimizes
$$ \mathcal{L}(\theta) = \mathcal{L}_{\text{task}}(h(\tilde{x}), y) + \beta\, \lVert \tilde{x} - x \rVert^2 - \lambda\, \mathcal{L}_{\text{trip}}(e_\phi(\tilde{x})). $$
Here $h$ is a frozen downstream task model trained on clean data, $y$ is its target (for example future head orientation), the distortion term with weight $\beta$ keeps outputs usable by consumers that never retrain (the distortion bound idea of GAP), $\lambda$ sets the privacy strength, and $\mathcal{L}_{\text{trip}}$ is the adversary's triplet loss. Sweeping $\lambda$ traces a privacy–utility curve. Freezing $h$ is deliberate: sanitized data must remain useful to models trained on ordinary gaze.
One variant addresses a real tension. The individual eye–head lag ranges from 0.08 to 0.2 seconds across people (Jin et al.), so it is both useful and partly identifying. In this variant, following CDAN, the adversary receives the task prediction alongside its features. The intended effect is that the sanitizer is penalized only for identity information beyond what the task output already explains. Whether this helps or simply leaks identity through the task channel is an empirical question, and it is reported either way.
Baselines
The naive reference is the raw unsanitized stream. On Jin et al.'s 360-degree dataset, it fixes the attacker ceiling and the utility ceiling: gaze-augmented prediction reached 96% tile accuracy against 92% for head-only. The head-only figure is the utility floor. A sanitizer that falls to 92% has destroyed the reason to stream gaze at all.
The strongest existing streaming defense is linearly weighted smoothing from Wilson et al.. It reduced re-identification from 67.3% to 14.1% while keeping 80–90% task success. We reimplement it, sweep its window, and plot it on the same privacy–utility axes using the same fresh attackers, so no method is scored against an attacker it was tuned against.
The strongest learned scanpath defense is the reinforcement-learning bottleneck of Fuhl et al. On ETRA 2019 it cut subject accuracy from 93% to 15% (chance 12%) while stimulus accuracy fell from 96% to 83%. A differential-privacy baseline at equal privacy kept only 30%. We compare on ETRA 2019 using their protocol, and additionally with a fresh open-set attacker.
Feature-level differential privacy from Steil et al. is included on aggregated features. At $\epsilon = 15$ it gave 54% gender accuracy and 6.4% re-identification. It cannot serve coordinate-level tasks, which we state rather than force a comparison.
The ablation that isolates the key ingredient is an identical causal sanitizer trained with a gradient-reversal closed-set user classifier, in the style of Wang et al. and GAP, under the same architecture, data, step budget, and hyperparameter search budget of 30 trials per method. Beating it means the following: at a matched utility level, re-identification of held-out users by a fresh triplet attacker is lower by more than the seed-to-seed spread over five seeds. A second ablation removes episodic user sampling, and a third swaps the causal sanitizer for a bidirectional one to price the latency constraint.
Why not just…?
Why not just tune smoothing harder? Smoothing is a real threat on black-box metrics. It is cheap and reached 14.1% re-identification in Wilson et al. Its costs are lag, which works directly against a lead of only 0.08–0.12 seconds, and determinism, which Wilson et al. note leaves it open to attackers who model the filter from exemplars. Our experiments include a white-box attacker that knows the filter or sanitizer and trains on matched pairs. If smoothing matches us on both lag-sensitive utility and white-box attacks, we report that the learned method is unnecessary.
Why not just use gradient reversal against a user classifier? This is the obvious transfer, and it is the central ablation. The argument against it is specific: fooling an N-way classifier does not imply removing per-person geometry for new people. No reported result on Lacuna tests closed-set adversarial gaze sanitizers against fresh open-set attackers. Fuhl et al.'s zero-shot transfer to WPL and DOVES is reported only qualitatively. If the closed-set version is just as good, the core claim fails.
Why not just add differential privacy noise? On scanpaths, equal-privacy DP left 30% stimulus accuracy versus 83% for a learned method (Fuhl et al.). Gaussian noise also hurt user satisfaction in Wilson et al. Temporally correlated DP (Günlü et al.) is the stronger form and offers formal guarantees our method lacks. We are explicit that our guarantee is empirical, measured against the strongest attackers we can build.
Why not just run the task on-device and never release gaze? For a single fixed task this is the correct answer, and we concede it. The proposal targets platforms where several applications, or a vision-language assistant using fixation history (Wilson et al.), need the stream itself. We test generality by checking utility on a task the sanitizer was not trained for.
Proposed Experiments
The primary testbed is Jin et al.'s 120 Hz head-and-gaze dataset: 100 participants and 27 clips. Users are split 60/20/20, and attackers and evaluation use only the held-out 20. The attacker enrolls each person from some clips and identifies them on others, reporting top-1 re-identification and verification equal error rate. The attackers include a Strohm-style triplet network, a recurrent classifier, and a white-box inverse model. Utility is measured as viewport prediction error, tile accuracy, and the recovered eye–head lag per person. ETRA 2019 provides comparison against Fuhl et al., and Wilson et al.'s interactive VR data, if obtainable, tests task success.
To probe generality, a sanitizer trained only for viewport prediction is applied, frozen, to ADT fixation histories for object identification. ADT's baseline accuracy rises from 10.3% to 24.8% with gaze history. We also measure whether sex and personality inference drop as a side effect, which is not guaranteed.
The decisive check is a single plot per dataset of fresh-attacker re-identification against utility. The proposal succeeds if the metric-adversary curve lies strictly below both the smoothing curve and the closed-set gradient-reversal curve in the region where tile accuracy stays above the 92% head-only floor. It fails if the white-box attacker restores re-identification near the raw-stream level.
References
- Learning User Embeddings from Human Gaze for Personalised Saliency Prediction
- Generalizing Eye Tracking With Bayesian Adversarial Learning
- Generative Adversarial Privacy
- Reinforcement Learning for the Privacy Preservation and Manipulation of Eye Tracking Data
- Where Are You Looking?: A Large-Scale Dataset of Head and Gaze Behavior for 360-Degree Videos and a Pilot Study
- Episodic Training for Domain Generalization
- Conditional Adversarial Domain Adaptation
- Privacy-Preserving Gaze Data Streaming in Immersive Interactive Virtual Reality: Robustness and User Experience
- Privacy-aware eye tracking using differential privacy
- Differential privacy for eye tracking with temporal correlations
- Eye Gaze as a Signal for Conveying User Attention in Contextual AI Systems