Abstract
Most visual saliency computing methods build models based on the content of an image without considering the colorized effects. Biologically, human attention can be significantly influenced by color. This study firstly investigates the sole contribution of colors in visual saliency and then proposes a bio-driven saliency detection method with a color factor. To study the color saliency despite the contents, an eye-tracking dataset containing color images and gray-scale images of the same content is proposed, collected from 18 subjects. The CIELab color space was selected to conduct extensive analysis to identify the contribution of colors in guiding visual attention. Based on the observations that some particular colors and combinations of color blocks can attract much attention than others, the influence of colors on visual saliency is represented computationally. Incorporating the color factor, a novel saliency detection model is proposed to model the human color perception prioritization, and a deep neural network model is proposed for eye fixation prediction. Experiments validate that the proposed bio-driven saliency detection models make substantial improvements in finding informative content, and they benefit the detection of salient objects which are close to human visual attention in natural scenes.
Introduction
When observing a scene, such as a bright red poppy in a green field, human attention is immediately drawn to the flower. A fundamental question in computer vision is whether this attraction is due to the object's semantic content (the shape of the petals) or its raw biological impact (the color red). Most existing saliency detection models focus on high-level features like shapes, textures, or semantic objects (faces, cars), often neglecting the intrinsic biological influence of color.
A major challenge in quantifying this effect has been the lack of specialized data. Standard eye-tracking datasets typically provide only color images, making it impossible to disentangle the influence of meaning from the influence of color. This research addresses this gap by quantifying the "sole contribution" of color to human attention and developing models that prioritize human color perception.
Methods and Dataset
The researchers developed a multi-stage approach to integrate color factors into saliency detection.
The Color-Gray Eye-Tracking Dataset (CGed)
To isolate the color factor, the study introduced the CGed dataset, which contains 500 pairs of brightly colored natural images and their grayscale counterparts. By recording the eye fixations of 18 subjects on both versions, the researchers could calculate a "Difference in Attention Score" (DAS). This metric effectively isolates the attention elicited purely by color, independent of the image's structural content.
Computational Modeling
Statistical analysis across various color spaces revealed that the CIELab space—specifically the 'b' component (yellow-blue axis)—correlates most strongly with human attention. Based on these findings, two novel models were proposed:
- RNCw (Region Contrast with color weights): An enhanced region-based model that uses weighted color distances to rank salient regions.
- APNet (Adaptive Priori attention weighting Network): A deep learning architecture utilizing a two-stream VGG-16 backbone. It includes a unique priori attention weighting subnetwork that adjusts the final saliency map based on known biological color sensitivities.
Key Results
The proposed bio-driven models significantly outperform state-of-the-art (SOTA) methods like SALICON and CASNet.
- Fixation Prediction: On the CGed dataset, APNet achieved superior performance across metrics such as AUC-Judd (0.83) and NSS (1.76). Notably, APNet outperformed a version of itself without the color-weighting subnetwork, confirming the value of the color factor.
- Salient Object Detection: The RNCw model demonstrated improved accuracy in object detection, reducing the Mean Absolute Error (MAE) on the ECSSD dataset from 0.188 (baseline) to 0.173.
- Qualitative Performance: Visual analysis shows that APNet is particularly effective at identifying small, vibrant objects that humans notice but traditional content-heavy models often overlook.
Conclusion and Future Work
This study confirms that human attention is influenced by a "color prioritization effect" that persists even when semantic content is ignored. By leveraging the CIELab color space and the new CGed dataset, the researchers have created more human-like artificial attention systems.
Future research will focus on expanding the CGed dataset to include more diverse scenes and integrating other biological cues, such as depth information or motion, into the saliency models. These advancements could improve downstream applications like automated video segmentation and object proposal generation.
Summary updated