Abstract
This paper introduces VisAlign, a benchmark designed to quantify AI-human visual alignment. By incorporating a novel "Uncertain" category with gold-standard human annotations, the authors evaluate how vision models handle ambiguity compared to human perception. Results indicate that current models are significantly overconfident, and that visual alignment serves as a critical indicator for AI reliability and safety.
Introduction
As deep learning models transition into real-world applications, the discrepancy between machine and human perception poses significant safety risks. Humans naturally tend to abstain from classification when presented with ambiguous, blurry, or corrupted visual data. In contrast, AI models are notorious for producing overconfident, erroneous predictions on such samples.
While existing research focuses on robustness (handling noise) or Out-of-Distribution (OOD) detection, the "middle ground" of uncertainty—where images are partially recognizable but ambiguous—remains under-explored. VisAlign addresses this gap by benchmarking models against human-derived "gold labels" across a spectrum of visual clarity, arguing that models perceiving the world like humans are more likely to behave safely in society.
The VisAlign Benchmark
The VisAlign dataset focuses on 10 mammal classes chosen for their clear biological definitions and human familiarity. It categorizes 13,400 images into three primary groups across eight distinct categories:
- Must-Act: Clear, photo-realistic images (including those with adversarial noise or spurious backgrounds) that humans easily classify.
- Must-Abstain: Images that clearly do not belong to the target classes or are non-photo-realistic (e.g., drawings or hybrid animals).
- Uncertain: Images subjected to varying levels of cropping and synthetic corruption (e.g., blur, brightness).
Unlike existing benchmarks such as ImageNet-C or OpenOOD, VisAlign is the only dataset to satisfy four essential requirements: clear class definitions, human familiarity, coverage of diverse realistic scenarios, and gold human labels for every sample.
Methodology
To establish ground truth for the "Uncertain" group, the researchers collected 134 annotations per image via Amazon Mechanical Turk. This generated a "gold human ratio"—a probability distribution reflecting how many humans would classify the image versus how many would abstain.
Alignment is measured using the Hellinger distance to compare the model’s output distribution against the human gold ratio. Additionally, the authors introduced a Reliability Score ($RSc$) that penalizes incorrect predictions while rewarding correct actions and justified abstentions. This score helps determine if a model is "safe" for sensitive deployments where the cost of an error is high.
Results and Discussion
The researchers benchmarked five popular architectures (including ViT and ConvNeXt) and seven abstention methods. Key findings include:
- Abstention Strategy Over Architecture: The choice of how a model decides to abstain (e.g., K-Nearest Neighbors vs. Softmax probability) has a more significant impact on alignment than the underlying model architecture. KNN-based methods generally achieved superior alignment by capturing fine-grained features.
- The Overconfidence Trap: Most models failed to maintain positive reliability scores when the cost of an error was high. This highlights that current models are not yet reliable enough for high-stakes ambiguous situations.
- Architecture Trends: Transformers (like ViT) showed improved alignment when pre-trained on large datasets, likely because they capture global features better than traditional CNNs.
Conclusion and Future Work
VisAlign demonstrates that visual alignment is a vital proxy for AI safety. Models that "see" more like humans are less likely to make catastrophic errors. While currently limited to 10 mammal classes and synthetic corruptions, the VisAlign framework provides a foundation for future research into human-aligned perception in complex tasks like object detection, segmentation, and the mitigation of social biases in AI.
Summary updated