Abstract
Distinguishing edges caused by a change in depth from other types of edges is an important problem in early vision. This study investigates the performance of humans and computer vision models on this task using spherical imagery with ground-truth LiDAR range data. By comparing various computational models for classifying depth from non-depth edges in small image patches, researchers achieved a peak performance of 86% with a convolutional neural network (CNN). Interestingly, human performance was found to be lower than the CNN, and while responses were correlated, humans and CNNs appear to weight low-level visual features differently, suggesting divergent strategies for solving the same visual problem.
Introduction
When navigating a visual environment, the human brain instantly distinguishes between the edge of a building against the sky (a depth edge) and a shadow on a sidewalk (a non-depth edge). This "edge classification" is a fundamental building block of 3D vision. However, defining ground truth for this task has historically relied on subjective human labeling. This research utilizes objective physical data to benchmark how well deep neural networks and humans can identify these boundaries using only local information.
Methodology and the SYNS Dataset
The study utilized the Southampton-York Natural Scenes (SYNS) dataset, which provides high-resolution panoramic imagery paired with precise LiDAR range measurements. By aligning 2D images with 3D depth maps, the researchers could mathematically define a "depth edge" as a physical jump in distance.
For the computational side, a CNN was trained on 200,000 edge patches. For the human side, eight participants classified 800 of the same patches in a psychophysical experiment. To ensure a fair comparison, patches were presented at the same angular size as they would appear in a real-world scene, forcing both systems to rely strictly on local visual cues.
Comparative Performance and Insights
The CNN significantly outperformed humans on local patches, achieving 86% accuracy compared to the human average of roughly 70-75%. This suggests that local image patches contain statistical information that humans do not fully exploit in isolation.
Key findings regarding visual strategies include:
- The Elevation Cue: One of the strongest predictors of a depth edge is its vertical position. Depth edges, like rooftops, are statistically more likely to occur above the horizon. The CNN exploited this "height" information effectively, whereas humans did not.
- Feature Weighting: Humans rely heavily on luminance contrast and orientation. In contrast, the CNN was more sensitive to color cues (specifically blue-yellow contrast, likely identifying sky-object boundaries) and texture.
- Divergent Strategies: Humans showed higher consistency with each other than with the CNN. This indicates that humans share a specific set of visual biases or heuristics that differ from the statistical shortcuts learned by the AI.
Conclusion
While CNNs currently surpass human performance in classifying edges within tiny, isolated patches, this likely reflects the CNN's ability to exploit subtle local statistics like elevation and color. In natural viewing, humans benefit from global context and eye movements that were excluded from this local task. Future work will explore how top-down scene context and more complex architectures might bridge the gap between biological and artificial vision strategies.
Summary updated