Introduction
Modern computer vision foundation models (FMs) like DINOv2 and CLIP are trained on vast datasets of natural images—the same visual statistics that have shaped human evolution. This raises a fundamental question: do these models converge on the same low-level visual solutions as the human visual system (HVS)? This paper investigates whether AI models share human-like "bottlenecks," such as specific spatial frequency sensitivities and contrast masking effects.
Methodology
To evaluate these models, the researchers treated neural networks as human subjects in a psychophysics lab. They utilized synthetic stimuli, primarily Gabor patches and band-limited noise, to test three core characteristics:
- Contrast Detection: The minimum signal required to perceive a pattern against a flat background.
- Contrast Masking: The difficulty of perceiving a pattern when obscured by a "masking" background.
- Contrast Constancy: The perceived intensity of high-contrast patterns across different scales.
The study evaluated 45 models (including DINO, DINOv2, OpenCLIP, and SAM) by measuring the cosine similarity ($S_{ac}$) between feature vectors of test and reference images. These responses were then compared against the castleCSF gold-standard human vision model.
Key Insights
The study revealed several critical points of alignment and divergence:
- Convergence in Masking: The strongest agreement between AI and humans occurred in contrast masking. This suggests that the ability to distinguish signal from background noise is a universal requirement for any effective vision system.
- DINOv2's Human-like Vision: DINOv2 emerged as the most human-aligned model. Its self-supervised training objective appears to capture fundamental aspects of visual representation more effectively than other methods.
- The Sensitivity Gap: AI models generally exhibit significantly lower sensitivity to low-contrast patterns compared to humans, often failing to detect details that are easily visible to the human eye.
Quantitative Results
The researchers found that while some models like DINOv2 and SD-VAE show hints of the human-like "peak" in spatial frequency sensitivity, many others, such as OpenCLIP, display jagged and inconsistent responses.
Quantitatively, DINOv2 variants consistently achieved the highest Spearman correlations with human psychophysical data across nine different test types, as shown in the performance summary above. In contrast, models like OpenCLIP showed high inconsistency across different spatial frequencies.
Limitations and Future Directions
A primary limitation identified is the "sRGB bottleneck." Most models are trained on 8-bit web images, which lack the full luminance range processed by the human eye, likely hindering the learning of true low-light sensitivity. Furthermore, it remains unclear whether the success of models like DINOv2 is a result of the Vision Transformer (ViT) architecture or the specific training recipe. Future research could explore how task-specific fine-tuning affects these low-level visual characteristics.
By establishing this bridge between psychophysics and deep learning, the study provides a robust benchmark for measuring the foundational "understanding" of visual building blocks in AI.
Summary updated