Abstract
Cross-domain gaze estimation is a critical task for applications in AR/VR and human-computer interaction, yet deep learning models often struggle to generalize to unseen environments. This paper introduces Gaze Frontalization-based Auxiliary Learning (GFAL), a novel framework that improves model robustness by incorporating gaze frontalization as an auxiliary task. By teaching the model to transform any input gaze into a "frontalized" view (looking directly at the camera), the system learns fundamental geometric features rather than dataset-specific noise. GFAL demonstrates significant performance gains, outperforming existing state-of-the-art domain generalization methods by up to 23.53% on benchmark datasets.
Introduction
Gaze estimation involves predicting the direction of a person's gaze from eye or face images. While current models perform well within the same dataset, their performance often degrades significantly when applied to "unseen" data with different lighting, camera angles, or subject ethnicities. This is primarily because gaze estimation is a continuous regression task with infinite possible Euler angles, making it prone to overfitting.
The challenge of cross-domain gaze estimation lies in making a model learn the intrinsic geometry of the eye. To address this, we propose the GFAL framework, which uses the concept of "gaze frontalization" as a geometric constraint to guide the learning process.
Methodology: The GFAL Framework
The core of the GFAL framework is the integration of an auxiliary task—gaze frontalization—into the main gaze estimation pipeline. This process acts as a regularizer, forcing the network to extract features that are consistent with the underlying physical rotation of the eyeball.
1. Feature Extraction and Frontalization
The framework begins with a standard backbone (e.g., ResNet-18 or ResNet-50) that extracts a feature vector $z$ from the input face image $x$. This feature is then used by the Gaze Frontalization Module to generate a "warping field" $M$. This field defines how to transform the original input image into a "frontalized" version $x_{fro}$, where the subject appears to be looking directly at the camera (gaze angle $(0,0)$).
2. Consistency Losses
To ensure the validity of the frontalization, the framework employs a feedback loop using consistency losses:
- Gaze Consistency: The frontalized image $x_{fro}$ is fed back into the gaze estimator. The model is penalized if the predicted gaze for this image deviates from the target $(0,0)$.
- Head Orientation Consistency: The model must also ensure that the head orientation in the frontalized image remains consistent with the original input, preventing the network from simply rotating the entire head to achieve a frontal gaze.
3. Flexibility
A key advantage of GFAL is its "plug-and-play" nature. While the primary implementation uses image warping, the framework can accommodate other frontalization techniques such as 3D reconstruction or gaze redirection.
Key Insights
- Frontalization as a Universal Anchor: Unlike arbitrary gaze angles, the frontal gaze $(0,0)$ is a universal constant across all datasets. Using it as a target for an auxiliary task provides a stable "anchor" for learning.
- Task Homogeneity: Gaze frontalization and gaze estimation are intrinsically related (homogeneous) as they both depend on eyeball rotation and head pose. This makes frontalization an ideal auxiliary task for regularization.
- Simplicity in Training: Frontalization is a simpler task than general estimation because the target is fixed. This simplicity helps stabilize the training of the more complex primary regression task.
Experimental Results
The GFAL framework was evaluated using four major gaze estimation benchmarks: ETH-XGaze, Gaze360, MPIIGaze, and EyeDiap. Models were trained on a source domain (e.g., ETH-XGaze) and tested on a target domain (e.g., MPIIGaze) to measure cross-domain generalization.
- Performance Gains: GFAL achieved a 23.53% improvement over the baseline on the ETH-to-MPII task.
- State-of-the-Art Comparison: The framework outperformed existing Domain Generalization (DG) methods and remained competitive with Unsupervised Domain Adaptation (UDA) methods, despite UDA methods having access to unlabeled target data during training.
- Backbone Robustness: Improvements were consistent across different network architectures, including ResNet-18 and ResNet-50.
Conclusion
GFAL provides a robust solution for cross-domain gaze estimation by embedding the gaze frontalization process as an auxiliary learning task. By focusing on the universal geometry of a frontal gaze, the model learns features that generalize far better to unseen environments. Future work may involve extending this framework to 3D geometry for even more precise frontalization and exploring the computational trade-offs of the auxiliary modules.
Summary updated