Abstract
Appearance-based gaze estimation often suffers from poor generalization across different domains due to overfitting in the final fully connected (FC) layers. This paper introduces the Analytical Gaze Generalization (AGG) framework, which replaces the over-parameterized FC layer with a geometrically-constrained Geodesic Projection Module (GPM). By leveraging the inherent spherical geometry of gaze, the GPM reduces the parameter count from thousands to just ten, significantly enhancing cross-domain robustness. Experimental results across four major benchmarks demonstrate up to a 35% improvement in cross-domain accuracy, providing a stable and plug-and-play solution for robust gaze estimation.
Introduction
Appearance-based gaze estimation—predicting where someone is looking based on a camera image—is notoriously fragile. A model trained in a well-lit lab often fails when deployed in real-world environments like homes or offices. The researchers identify a specific culprit: the final Fully Connected (FC) layer. Standard models map high-dimensional features (e.g., 512D) to a 3D gaze vector using an FC layer with thousands of parameters. This high capacity allows the model to memorize gaze-irrelevant details such as lighting, skin tone, or camera quality, leading to severe overfitting.
Methods
The authors propose the Analytical Gaze Generalization (AGG) framework to address this. It consists of two primary components:
- Geodesic Projection Module (GPM): Instead of a massive FC layer, the GPM uses manifold learning (specifically Isomap) to project high-dimensional features into 3D space. This is based on the physical observation that the distance between features in the latent space should be proportional to the angular difference in gaze. The GPM replaces thousands of parameters with just 10 learnable parameters for physical rotation and scaling.
- Sphere-Oriented Training (SOT): To overcome the non-differentiable nature of Isomap, the authors introduced an Isometric Propagator—a small MLP that simulates the Isomap function during training. This allows the model to optimize its feature extractor so that features naturally align with spherical geometry.
Key Insights
The core innovation lies in recognizing the Geometry of Gaze. Gaze-related features in a well-trained model naturally form a manifold mirroring the physical world. By replacing a 1500+ parameter "black box" with a 10-parameter geometric alignment module, the model is forced to ignore domain-specific noise that does not fit the spherical geometry of eye rotation. This "less is more" approach injects analytical physical constraints directly into the network architecture.
Results and Evaluation
The framework was evaluated across four major benchmarks: ETH-XGaze, Gaze360, MPIIFaceGaze, and EyeDiap. Testing across 12 cross-domain scenarios, the AGG framework consistently outperformed state-of-the-art methods, improving accuracy by up to 35.79% in some cases. Furthermore, the GPM-based models exhibited significantly more stable error rates across training epochs compared to standard models. The method is "plug-and-play," showing consistent improvements across various backbones like ResNet-18, ResNet-50, and VGG16.
Conclusion
The AGG framework demonstrates that injecting analytical physical constraints into network architectures can drastically improve generalization. While the current training relies on an MLP-based "Isometric Propagator" as a compromise for differentiability, the results highlight the potential of moving away from over-parameterized linear layers in regression tasks with clear physical geometries, such as human pose estimation.
Summary updated