When people concentrate on a demanding task at the center of their gaze, they become markedly worse at seeing faint detail in the periphery. Foveated rendering exploits this: it draws high detail only where the eye points and degrades the rest of the image. Towards Attention-aware Foveated Rendering showed that a hard central task can raise peripheral contrast-detection thresholds by up to 4x and nearly triple the blur that goes unnoticed. That model treats the periphery as uniform, as if a face, a line of text, and a patch of foliage at the same eccentricity were suppressed equally. Saliency research says otherwise. Faces, eyes, and text pull gaze whatever their contrast or size (Predicting eye fixations using convolutional neural networks), and features from object-recognition networks predict most of where people look (DeepGaze II).
This note proposes a combined psychophysics and modeling study of what happens where these two findings meet. We hypothesize that a peripheral semantic attractor partly restores the sensitivity lost to central load, and that the restoration follows the rule physiology reports for spatial attention. That rule is a uniform spatial gain: a boost applied to everything at a location, across all feature channels, without sharpening tuning for any particular feature. If this is right, a face in the periphery should protect not only face-like detail but also an unrelated grating or a blur artifact drawn next to it. The extent of this protection should shrink with distance at a rate we can measure. That result would yield a perceptual model and a renderer that degrade the periphery aggressively everywhere except near attractors.
Core Claim
Our claim is that load-induced peripheral suppression is not uniform across a natural scene. A semantic saliency map, smoothed by a spatial pooling kernel, modulates it, and the modulation is feature-agnostic within that pool. Two physiological results motivate this. Pre-saccadic neural enhancements in marmoset area MT found that attention before a saccade raised MT firing by a median of about 14% across the whole direction-tuning curve, with no change in tuning width. Spatial Distribution of Attentional Modulation at Columnar Resolution in Macaque V4 found that spatial attention enhanced broad, stimulus-nonspecific signals spanning millimeters and left fine stimulus-selective columns unchanged. Both point to a location-wide volume knob rather than a feature filter.
A competing account comes from feature-based attention, which increases response gain for an attended feature across the visual field (Herrmann et al., 2011). It predicts that any protection a face confers should be specific to the features the face carries. The two accounts make different predictions for a probe whose orientation and spatial frequency are unrelated to the attractor, and that difference is what makes the study decisive.
We formalize the claim with one equation. Let $S_0(x)$ be the load-free contrast sensitivity at visual location $x$, given by an eccentricity model such as StelaCSF. Let $g_L \in (0,1]$ be the uniform suppression factor that central load $L$ imposes, fitted as in the attention-aware model. Let $A(x) \in [0,1]$ be a semantic attractor map smoothed by a Gaussian of width $\sigma$. The predicted sensitivity is
$$ S(x, L) = S_0(x)\,\big[\,g_L + \alpha\,(1 - g_L)\,A(x)\,\big], $$
where $\alpha \in [0,1]$ is the fraction of suppressed sensitivity that an attractor restores. Uniform spatial gain corresponds to $A(x)$ being independent of the probe's features. With $\alpha = 0$ the equation reduces to the existing load model. The form predicts that the benefit of an attractor grows with load, because there is more suppression to undo. A model that adds saliency without reference to load does not make that prediction.
Research Program
The project has three parts. The first is a covert-attention psychophysics experiment that extends the rapid serial visual presentation paradigm of Krajancich and colleagues, in which letters flash at fixation at three difficulty levels. While participants perform that task, they detect peripheral probes embedded in natural photographs drawn from MIT1003 and the SALICON dataset. An eye tracker aborts any trial with a fixation break, so all effects are covert. Probes are either Gabor patches at orientations and frequencies unrelated to nearby content, or local blur. Each probe sits at a controlled distance from a peripheral item at matched eccentricity. That item is an intact face, an intact text string, a phase-scrambled version of the same item with matched luminance contrast and spectrum, or a non-semantic object of equal low-level saliency.
The second part fits the model. $A(x)$ comes from DeepGaze II with the center-bias prior removed, so it reflects content rather than screen position. We estimate $\alpha$ and $\sigma$ per participant, and we also fit a feature-specific variant in which $\alpha$ depends on whether the probe shares orientation and frequency energy with the attractor.
The third part is an application. A foveated renderer allocates detail according to $S(x,L)$, and we test it with blur-detection and preference judgments on the original four scenes plus new scenes containing peripheral people and signage.
Two strands of work constrain the design. Attention arithmetic is not settled: V1 shows additive boosts (Additive Effects of Attention and Stimulus Contrast in Primary Visual Cortex), and V4 laminae mix contrast gain and response gain (Wang et al., 2023). We therefore measure thresholds at several pedestal contrasts and test whether the attractor benefit acts as a multiplicative or a contrast-dependent change. We do not assume one. The second constraint is that ImageNet-trained features under-represent text (DeepGaze II). Text attractors therefore serve as a deliberate stress test of the saliency component.
Prior generative and foundation work is adjacent but different. Horikawa & Kamitani showed that attention biases representations like a contrast boost across a network hierarchy, but only for full-field superimposed images and with attention cued by instruction. Attend to Anything predicts where people look across many datasets but has no notion of load or sensitivity. Neither asks how semantic capture changes what is visible at a location.
Baselines
The naive reference is a pure eccentricity model, StelaCSF with no load term. Krajancich and colleagues already showed it overestimates peripheral sensitivity under load. It shows how much any attention term matters at all.
The strongest existing competitor is the attention-aware load model of Towards Attention-aware Foveated Rendering, our equation with $\alpha = 0$. That paper reports threshold increases of up to 4x under high load, nearly tripled blur tolerance in the City scene, and better accuracy than StelaCSF at $p < 0.01$ in most cases. It does not report a threshold-prediction error in a form we can quote, so we refit it on our own data. Both models receive the same participants, trials, and cross-validation folds, and per-participant fitting is limited to the same number of free parameters plus the two we add. Beating it means a lower held-out threshold-prediction error that is significant across participants, concentrated at probes near attractors, and present under medium and high load.
A second strong competitor is saliency-weighted foveation without load interaction. Here sensitivity is $S_0(x)\,[g_L + \beta A(x)]$ with $\beta$ shared across all load levels, and $A(x)$ is the same DeepGaze II map. DeepGaze II reaches 88% AUC and 77% shuffled AUC on MIT300 and captures about 87–88% of explainable information on MIT1003 (Using Deep Features to Predict Where People Look). We found no reported result on Lacuna for saliency-weighted foveation predicting perceptual thresholds. This model gets the same saliency input and fitting budget as ours, so it isolates the load interaction.
Two ablations isolate the key ingredients. The first replaces DeepGaze II with a low-level contrast saliency map, which tests whether semantics, not conspicuity, drive the effect. The second is the feature-specific gain variant, which tests uniformity.
Why not just…?
Why not just weight the foveated renderer by a saliency map? This is the most serious threat, and the second baseline tests it directly. If a load-independent saliency weight fits as well as the interaction model, the gain-field framing is unnecessary and we will report that. The interaction predicts something the additive version cannot: the attractor benefit should be near zero under low load and largest under high load. A flat benefit across load levels would favor plain saliency weighting.
Why not just keep the uniform load model, since peripheral attractors are rare? In the four test scenes of the original study they may be rare. In social, urban, and interface content, faces and text are common in the periphery, and those are exactly the scenes where blur near a person is most objectionable. The rendering study measures whether ignoring attractors produces visible artifacts there that the uniform model predicts should go unseen.
Why not just assume the protection is face-specific feature gain? Feature-based gain is real (Herrmann et al., 2011), so this alternative is plausible. Its practical consequence would be that rendering only needs to preserve the attractor's own features. The unrelated-Gabor probes settle the question. Equal restoration for related and unrelated probes supports uniform spatial gain, as in MT and V4. Restoration only for related probes supports feature gain, and in that case we would keep the feature-specific model.
Why not just attribute the effect to low-level contrast at the attractor? Phase-scrambled controls with matched spectrum and the low-level saliency ablation address this. If scrambled faces protect as much as intact ones, the effect is ordinary conspicuity, not semantic selection.
Decisive Checks
The study succeeds if three results hold together. First, intact faces and text, but not their scrambled counterparts, lower thresholds for nearby probes under high load, and the benefit grows significantly from low to high load. Second, the benefit is statistically indistinguishable for feature-unrelated Gabors and for blur, and it falls off with a fitted $\sigma$ that is stable across scenes. Third, the fitted model predicts held-out thresholds better than both the load-only and the saliency-only baselines.
On the application side, a renderer driven by $S(x,L)$ should degrade more pixels than the load-only renderer while keeping blur-detection rates near attractors no higher. Pre-registered failure conditions are a flat load interaction, feature-specific restoration, or no semantic-over-scrambled advantage. Each would falsify a named part of the claim.
References
- Towards Attention-aware Foveated Rendering
- Predicting eye fixations using convolutional neural networks
- DeepGaze II: Reading fixations from deep features trained on object recognition
- Pre-saccadic Neural Enhancements in Marmoset Area MT
- Spatial Distribution of Attentional Modulation at Columnar Resolution in Macaque Area V4
- Feature-based attention enhances performance by increasing response gain
- SALICON: Saliency in Context
- Additive Effects of Attention and Stimulus Contrast in Primary Visual Cortex
- Laminar compartmentalization of attention modulation in area V4 aligns with the demands of visual processing hierarchy in the cortex
- Attention modulates neural representation to render reconstructions according to subjective appearance
- Attend to Anything: Foundation Model for Unified Human Attention Modeling
- Using Deep Features to Predict Where People Look