Everyone's line of sight is tilted a few degrees away from the geometric axis of their eye. The geometric axis, called the optical axis, runs through the center of the cornea and pupil. The line of sight, called the visual axis, runs from the point being looked at to the fovea, the sharp-vision pit at the back of the retina. The angle between them, the kappa angle, differs from person to person. It cannot be seen from outside because the fovea is hidden, so every absolute gaze tracker estimates it from a few calibration fixations. The estimated offset absorbs whatever systematic error the tracker made in measuring the optical axis. We argue that when the eye is reconstructed in 3D from only two to four cameras, as in headsets and small multi-camera rigs, the typical reconstruction artifacts do not average out. These artifacts are semi-transparent "floaters" spawned to explain moving corneal glints, and a blurred boundary between iris and white of the eye. They bend the recovered optical axis in a direction that depends on the camera layout, and calibration then records that bend as if it were the person's kappa. The fake kappa fails to transfer when head pose changes.
This note proposes to measure that confound and remove it. We build a sparse-view 3D eye model with three properties. The eyeball is shared and rigid across many frames of one person. The kappa angle is an explicit, separately estimated parameter. The eye region is fit with a structural similarity loss, which scores local patterns and edges rather than individual pixels. We then test on synthetic data with exact kappa labels whether this recovers true kappa rather than "kappa plus artifact bias." The first part of the claim is the most novel: calibration error in sparse-view eye reconstruction can be split into a true physiological offset and a view-dependent bias. The second part is that sharing the static eye across frames, together with a structure-preserving loss, shrinks the bias term.
Core Claim
For a subject calibrated on $K$ fixations with known targets, the fitted offset satisfies
$$\hat{\kappa} = \kappa + \frac{1}{K}\sum_{k=1}^{K} b(V, h_k),$$
where $\kappa$ is the true angular offset, $V$ is the camera configuration, $h_k$ is head pose at fixation $k$, and $b$ is the angular error of the reconstructed optical axis. If $b$ were zero-mean noise, calibration would be harmless. We hypothesize that under sparse views $b$ is systematic, because glint floaters and a blurred limbus (the iris boundary) shift the fitted pupil-iris geometry consistently toward the cameras. That makes $\hat{\kappa}$ wrong in a way that grows when test head poses differ from calibration poses. The prediction is falsifiable on GazeGene, which provides exact kappa per synthetic subject GazeGene. A method that truly separates the two terms should show kappa error that stays flat as the calibration and test head-pose distributions are pulled apart.
Research Program
The model builds on three lines of work.
GazeNeRF represents the face and the eyes as separate neural volumes and rotates the eye volume rigidly to redirect gaze. It supplies our articulation mechanism, but it trains on dense data, treats the optical axis as the line of sight, and uses pixel and perceptual losses. De^2Gaze splits static anatomy (eyeball center and radius) from dynamic pupil rotation, and samples image features only on pupil and iris contours to avoid glints. It supplies the static/dynamic split, but it is a discriminative regressor, and its authors state explicitly that it ignores kappa. RegNeRF regularizes sparse-view radiance fields by rendering unseen viewpoints and penalizing depth roughness. It constrains our design: its depth-smoothness penalty would wrongly smooth over the real depth step at the eyelid margin, so we do not apply it to the eye.
Our representation has two parts. A face volume is shared per subject. An explicit eyeball is modeled as a sphere of learnable radius and center, fixed per subject, carrying an iris/pupil texture and a thin specular corneal layer. Only a per-frame rotation varies. The kappa angle is a per-subject two-parameter rotation that maps the optical axis to the visual axis. It is fit only from the $K$ calibration fixations and never from appearance.
Because the eyeball is shared across hundreds of frames at different gaze angles, each frame's two to four views jointly constrain a single rigid object. The sparse-view problem thus becomes a many-view problem for the static anatomy. This cross-frame sharing is the key ingredient.
The eye-region loss combines $L_1$ with a structural similarity term, computed on a band around the rendered limbus. Structural similarity compares local mean, contrast, and correlation within small windows. Pixel losses penalize a one-pixel edge shift twice and drive the model toward blur Li et al.. A structural term keeps the limbus sharp, and a sharp limbus is what fixes the optical axis. This follows DiffNR, which used a 3D structural similarity loss to suppress artifacts in underconstrained sparse-view CT volumes. We move the idea from tomography to an articulated, specular eye.
We are explicit about one limit of the loss. Glints change local structure, so the structural term alone does not remove them. The specular corneal layer is what absorbs glints, and the structural loss is scored on the diffuse iris rendering.
Baselines
The naive reference is a per-frame sparse-view reconstruction with pixel $L_1$ loss and no shared eyeball, followed by the same kappa calibration. Vanilla sparse-view fitting is known to collapse badly: RegNeRF reports 8.68 dB PSNR for mip-NeRF on 3 DTU views against 18.89 dB for RegNeRF. This baseline tells us how large the bias term is when nothing is done about it.
The strongest learned competitor for absolute gaze after few-shot calibration is FAZE Few-Shot Adaptive Gaze Estimation. It reports 3.18° on GazeCapture with 3 calibration samples and 3.14° on MPIIGaze with 9. We also include a ResNet-50 appearance regressor with personal calibration: ETH-XGaze reports 4.5° uncalibrated and about 2° with 200 calibration samples. Both baselines are retrained on the same subject splits and receive the same $K$ calibration fixations, with matched tuning budgets. We count as beating them only if we match their error at $K \le 9$ while showing smaller degradation under head-pose shift between calibration and test. No reported result was found on Lacuna for kappa-angle estimation error on GazeGene, so that number will be established here.
A second geometric competitor is GazeNeRF with the same sparse views. To give it a kappa estimate, we add a post-hoc offset fitted to its rendered-eye optical axis.
Three ablations isolate the proposal's ingredients. The first removes cross-frame sharing, so the eyeball is fit per frame. The second swaps the structural term for pure $L_1$. The third removes the specular layer.
Why not just…?
Why not just fit ellipses to the pupil and limbus in each view and triangulate? This classical pipeline is a serious threat. With calibrated cameras it may recover the optical axis well on clean images. We will run it on the same views. We expect it to break under glints on the pupil edge, eyelid occlusion of the limbus, and the extreme poses of ETH-XGaze. De^2Gaze found that glints and shadows are exactly what degrade contour-based estimates. If triangulation shows equally small, pose-independent bias on GazeGene, the neural field is unnecessary, and we will report that.
Why not just fine-tune a 2D gaze network on a few calibration images? A 2D network's calibration offset also absorbs its own systematic errors, and it provides no optical axis to separate them from kappa. FAZE is our stand-in for this approach. The decisive test is the head-pose-shift experiment described below, not the average error.
Why not use differential gaze, which cancels kappa entirely? Differential estimation A Differential Approach for Gaze Estimation removes static person-specific offsets by predicting only gaze changes between frames. Absolute gaze still needs one reference, and that reference inherits the same bias. We include a differential variant anchored on a single calibration point for comparison.
Why not just use the usual $L_1$ plus structural similarity mix over the whole image? The mix itself is not our contribution. The ablation tests whether restricting the structural term to the limbus band, in combination with a separate specular layer, reduces axis bias beyond the whole-image default.
Proposed Experiments
On GazeGene we render each synthetic subject from 2, 3, and 4 virtual cameras. We report optical-axis error, visual-axis error, and kappa error in degrees. We also report the correlation of optical-axis error with camera placement, which is a direct measurement of the systematic bias $b$.
On ETH-XGaze we subsample the 18 cameras to 2–4 views. We calibrate on $K \in \{3, 9\}$ fixations drawn from a narrow head-pose range and test on a disjoint, wider range. The metric is absolute gaze error; ground-truth kappa does not exist for real subjects. We add image quality in the eye region, reported as SSIM and PSNR on held-out views.
The decisive check is this. As the gap between calibration and test head pose grows, the naive baseline's gaze error should rise, while ours should stay near flat. On GazeGene, our $\hat{\kappa}$ should approach true kappa as more frames are shared. If the naive baseline's bias turns out to be pose-independent, the central hypothesis is false.
A secondary check applies GazeGene's sim-to-real protocol. We test whether kappa priors learned from synthetic subjects reduce the number of calibration points needed on ETH-XGaze.
References
- GazeGene: Large-scale Synthetic Gaze Dataset with 3D Eyeball Annotations
- GazeNeRF: 3D-Aware Gaze Redirection with Neural Radiance Fields
- De^2Gaze: Deformable and Decoupled Representation Learning for 3D Gaze Estimation
- RegNeRF: Regularizing Neural Radiance Fields for View Synthesis from Sparse Inputs
- DenseFuse: A Fusion Approach to Infrared and Visible Images
- DiffNR: Diffusion-Enhanced Neural Representation Optimization for Sparse-View 3D Tomographic Reconstruction
- Few-shot Adaptive Gaze Estimation
- ETH-XGaze: A Large Scale Dataset for Gaze Estimation Under Extreme Head Pose and Gaze Variation
- A Differential Approach for Gaze Estimation