Introduction
Gaze estimation is a fundamental task for applications ranging from human-computer interaction to medical diagnostics. While appearance-based methods using deep learning offer flexibility with standard webcams, they often struggle with a "generalization gap" when moving from controlled training environments to diverse real-world settings. Traditional solutions rely on domain adaptation, requiring target-domain data. In contrast, PCFGaze focuses on domain generalization, leveraging the inherent physical geometry of eye movement to create a robust, "out-of-the-box" solution.
Methodology: The PCFGaze Framework
The core innovation of PCFGaze is the alignment of the neural network's feature space with the physical world. The authors discovered that the "gaze feature manifold" in a trained network mirrors the spherical nature of the eye. Specifically, the geodesic distance between features is proportional to the actual angular difference in gaze.
Physics-Consistent Feature (PCF) and Spherical Fitting
To exploit this, the framework introduces Physics-Consistent Features (PCF). Using manifold learning techniques like Isomap, high-dimensional gaze features are reduced to a 3D spherical shell. Instead of a standard numerical regressor, PCFGaze employs Spherical Fitting (SF)—an analytical method that projects features onto a sphere to calculate pitch and yaw using geometric principles.
PCF-Oriented Training
To make this process efficient and differentiable for training, the authors developed an Isometric Propagator. This small multi-layer perceptron (MLP) simulates the Isomap transformation, allowing a ResNet-18 backbone to be trained to produce features that naturally adhere to the physical spherical manifold.
Experimental Results
The framework was evaluated across four major benchmarks: ETH-XGaze ($D_E$), Gaze360 ($D_G$), MPIIFaceGaze ($D_M$), and EyeDiap ($D_D$). PCFGaze consistently outperformed state-of-the-art generalization methods across multiple cross-dataset scenarios.
- Accuracy Improvements: The model achieved up to a 29.23% error reduction in the transfer from MPIIFaceGaze to EyeDiap compared to baselines.
- Zero-Shot Generalization: Performance gains were achieved without any exposure to target-domain data during training, demonstrating the power of physical constraints.
- Training Stability: By replacing black-box regression with analytical projection, the model exhibited lower variance and higher stability across training epochs.
Conclusion
PCFGaze demonstrates that incorporating physical constraints into deep learning architectures can significantly enhance robustness and interpretability. By forcing features to follow the geometry of the physical task, the model avoids overfitting to domain-specific artifacts like lighting or camera resolution. Future work may extend this "physics-consistent" paradigm to other regression tasks governed by known physical laws, such as human pose estimation or object orientation tracking.
Summary updated