Abstract: Visual saliency prediction is a growing field with numerous applications, yet most models are evaluated on urban datasets containing clear, salient objects. This study evaluates top-performing algorithms on naturalistic scenes devoid of obvious subjects. We introduce and test a biologically-inspired model based on early visual processing in primates, finding that it performs competitively with complex engineered models in natural environments. Our results highlight significant performance gaps between standard and natural datasets and identify image statistics that predict human gaze agreement.
The Challenge of Naturalistic Saliency
In computer vision, "saliency" refers to predicting where a human will look first. While many models excel on benchmarks like the Bruce & Tsotsos dataset, these often feature "composed" images with clear focal points, such as a red stop sign or a person. In these contexts, high-level cognitive priors—like looking for faces—dominate.
This research investigates how these models perform in truly naturalistic settings, such as forests or fields, where visual textures and patterns drive attention rather than distinct objects.
Methods: The Second Order Contrast (SOC) Model
The researchers benchmarked 10 leading saliency algorithms against a model inspired by the primary visual cortex (V1) of primates. This Second Order Contrast (SOC) model utilizes a three-stage pipeline:
- Spectral Energy: Log-Gabor filters mimic neural responses to various orientations and scales.
- Divisive Normalization (DN): Neural responses are suppressed by neighboring activity to prevent signal saturation in high-contrast regions.
- Second Order Contrast: Spatial variance of the energy is computed to identify regions that stand out from their surroundings.
Evaluation was conducted using Shuffled AUC (sAUC) to account for "center bias"—the human tendency to look at the middle of an image regardless of content.
Datasets
Two primary datasets were utilized to compare performance across environments:
- DOVES: 101 grayscale naturalistic images (e.g., forests, textures) with 30,000 human fixation points. This dataset lacks obvious salient objects.
- Bruce & Tsotsos: A baseline of 120 color urban scenes containing clear objects.
Key Insights and Results
The study revealed a "natural penalty": saliency prediction is significantly harder in the wild. Performance on the DOVES dataset was markedly lower than on standard object-heavy datasets across all models.
- Simplicity vs. Complexity: The SOC model achieved a Shuffled AUC of 0.556 on DOVES, tying for the highest score. This suggests that in the absence of clear objects, low-level energy and contrast are the primary drivers of attention, making complex "object-detecting" logic less effective.
- Inter-Observer Agreement: Human observers agree far less on where to look in natural scenes (~0.55 agreement score) compared to urban scenes (~0.74).
- Predictive Statistics: The authors found that the kurtosis of DCT coefficients—a measure of image data sparsity—can predict the level of agreement among human observers regarding a scene's focal points.
Conclusion and Future Directions
The SOC model's success on naturalistic data demonstrates the power of biologically-inspired principles. However, its current grayscale-only implementation is a limitation for color-rich benchmarks. Future work should integrate these low-level statistical properties into more robust models and expand naturalistic benchmarking to prevent algorithmic over-fitting to urban photography.
Summary updated