TL;DR
FocusGate is a gated ensemble of "gaze-free" priors (like defocus, motion, and face detection) that improves video gaze prediction by knowing exactly when to trust its inputs. By using a per-frame reliability gate to filter out noisy defocus cues, it consistently boosts the performance of state-of-the-art supervised models across multiple domains with negligible latency overhead.
What Problem Does This Paper Address?
Where do people look when watching a video? While modern video saliency is dominated by massive, gaze-trained deep networks, there is still valuable signal in "gaze-free" priors—explicit cues like what’s in focus, where faces are, or how objects move.
The problem is that these priors aren't always reliable. A defocus map is a great predictor of gaze in a shallow depth-of-field shot (where the subject is sharp and the background is blurred), but in a deep-focus shot, the same estimator might just output texture noise. If you always fuse these priors into your model (unconditional fusion), the noise from unreliable frames can actually hurt overall performance. The authors set out to build a system that can "abstain" from making a prediction when a specific cue is likely to be wrong.
Methods
The authors introduce FocusGate, an ensemble of eight gaze-free priors (including center bias, face detection, motion, and DINO-based attention).
The core innovation is the Reliability Gate ($c_t$). Instead of looking at the ground truth (gaze data), the gate analyzes the shape of the defocus map ($D$) itself. It calculates three statistics:
- Spread: The difference between high and low probability values.
- Separation: How distinct the "in-focus" and "out-of-focus" modes are.
- Sharp Fraction: The percentage of the frame considered in-focus.
If the map is "bimodal" (clear subject vs. clear background), the gate opens. If it's "unimodal" (everything is sharp or the map is just noise), the gate closes, and the prior effectively abstains.
To handle these abstentions without adding complex parameters, they use midrank normalization. This ensures that if a prior (like a face detector) finds nothing, it produces a constant map that remains inert during the fusion process. FocusGate can be used as a standalone predictor or added to a supervised model ($M$) by simply summing their rank-normalized outputs.
Datasets
The researchers evaluated FocusGate across four distinct domains to ensure robustness:
- Hollywood-2 (H2): Professional film footage (primary test set).
- UCF-Sports: Action-heavy sports clips (transfer domain).
- DHF1K: Web-style videos (transfer domain).
- DIEM: A confirmatory fourth domain, pre-registered and scored only once after all parameters were frozen to prevent over-fitting.
The defocus estimators and gate thresholds were tuned using the CUHK blur detection dataset, entirely independent of any gaze-tracking data.
Key Insights and Ideas
- The Collapse Regime: The authors discovered that defocus estimators have a specific "collapse regime" where their performance drops to chance. Crucially, this regime is identifiable purely from the estimator's own output (see Figure 2). Gating works because it identifies these "chance" frames and discards them.
- Complementarity over Strength: Even though FocusGate alone isn't as strong as the latest 6.9-billion-parameter supervised models, it still improves them. This proves that gaze-free priors contain "residual" signals that even the largest models haven't fully absorbed.
- Efficiency: Because the gate relies on simple shape statistics, it is incredibly fast. Adding FocusGate to the NTIRE 2026 champion model (ViSAGE) adds only 1% to its total latency.
Key Results
FocusGate demonstrated impressive consistency across all tests:
- Universal Improvement: It improved the shuffled AUC (sAUC) of four different supervised models across all sixteen model-domain combinations tested.
- Converting Harm to Gain: On sports footage, where the defocus cue is particularly noisy, unconditional fusion lowered performance by 1.40 points. The reliability gate flipped this into a significant gain of 0.68 points.
- State-of-the-Art Boost: When added to ViSAGE, it achieved the highest scores in the study (e.g., 81.9 sAUC on film).
- Standalone Power: With a 16-frame causal mean, the ensemble alone outperformed older supervised models like TASED-Net and UNISAL on film and DIEM datasets.
Limitations and Future Work
The current reliability gate is highly specialized for defocus cues. The authors found that other cues, like motion or general sharpness, didn't exhibit the same clear "collapse to chance" behavior, meaning the current gating logic didn't provide the same benefits for them.
Future research will focus on identifying similar reliability statistics for other types of priors. Additionally, while FocusGate excels at ranking (sAUC), it doesn't inherently improve density-based metrics (like NSS) without a separate calibration step, as the priors primarily provide information about the order of importance rather than the absolute probability density of fixations.
Summary updated