VisAlign: Dataset for Measuring the Degree of Alignment between AI and Humans in Visual Perception

Jiyoung Lee, Seungho Kim, Seunghyun Won, Joonseok Lee, Marzyeh Ghassemi, James Thorne, Jaeseok Choi, O.-Kil Kwon, Edward ChoiarXiv 2023 · DOI · OpenReview

Abstract

This paper introduces VisAlign, a benchmark designed to quantify AI-human visual alignment. By incorporating a novel "Uncertain" category with gold-standard human annotations, the authors evaluate how vision models handle ambiguity compared to human perception. Results indicate that current models are significantly overconfident, and that visual alignment serves as a critical indicator for AI reliability and safety.

Introduction

As deep learning models transition into real-world applications, the discrepancy between machine and human perception poses significant safety risks. Humans naturally tend to abstain from classification when presented with ambiguous, blurry, or corrupted visual data. In contrast, AI models are notorious for producing overconfident, erroneous predictions on such samples.

While existing research focuses on robustness (handling noise) or Out-of-Distribution (OOD) detection, the "middle ground" of uncertainty—where images are partially recognizable but ambiguous—remains under-explored. VisAlign addresses this gap by benchmarking models against human-derived "gold labels" across a spectrum of visual clarity, arguing that models perceiving the world like humans are more likely to behave safely in society.

The VisAlign Benchmark

The VisAlign dataset focuses on 10 mammal classes chosen for their clear biological definitions and human familiarity. It categorizes 13,400 images into three primary groups across eight distinct categories:

  1. Must-Act: Clear, photo-realistic images (including those with adversarial noise or spurious backgrounds) that humans easily classify.
  2. Must-Abstain: Images that clearly do not belong to the target classes or are non-photo-realistic (e.g., drawings or hybrid animals).
  3. Uncertain: Images subjected to varying levels of cropping and synthetic corruption (e.g., blur, brightness).
The VisAlign dataset structure across Must-Act, Must-Abstain, and Uncertain groups.

Unlike existing benchmarks such as ImageNet-C or OpenOOD, VisAlign is the only dataset to satisfy four essential requirements: clear class definitions, human familiarity, coverage of diverse realistic scenarios, and gold human labels for every sample.

Comparison of VisAlign against existing benchmarks on four key requirements.

Methodology

To establish ground truth for the "Uncertain" group, the researchers collected 134 annotations per image via Amazon Mechanical Turk. This generated a "gold human ratio"—a probability distribution reflecting how many humans would classify the image versus how many would abstain.

Alignment is measured using the Hellinger distance to compare the model’s output distribution against the human gold ratio. Additionally, the authors introduced a Reliability Score ($RSc$) that penalizes incorrect predictions while rewarding correct actions and justified abstentions. This score helps determine if a model is "safe" for sensitive deployments where the cost of an error is high.

Results and Discussion

The researchers benchmarked five popular architectures (including ViT and ConvNeXt) and seven abstention methods. Key findings include:

  • Abstention Strategy Over Architecture: The choice of how a model decides to abstain (e.g., K-Nearest Neighbors vs. Softmax probability) has a more significant impact on alignment than the underlying model architecture. KNN-based methods generally achieved superior alignment by capturing fine-grained features.
  • The Overconfidence Trap: Most models failed to maintain positive reliability scores when the cost of an error was high. This highlights that current models are not yet reliable enough for high-stakes ambiguous situations.
  • Architecture Trends: Transformers (like ViT) showed improved alignment when pre-trained on large datasets, likely because they capture global features better than traditional CNNs.

Conclusion and Future Work

VisAlign demonstrates that visual alignment is a vital proxy for AI safety. Models that "see" more like humans are less likely to make catastrophic errors. While currently limited to 10 mammal classes and synthetic corruptions, the VisAlign framework provides a foundation for future research into human-aligned perception in complex tasks like object detection, segmentation, and the mitigation of social biases in AI.

Summary updated

Abstract

AI alignment refers to models acting towards human-intended goals, preferences, or ethical principles. Given that most large-scale deep learning models act as black boxes and cannot be manually controlled, analyzing the similarity between models and humans can be a proxy measure for ensuring AI safety. In this paper, we focus on the models' visual perception alignment with humans, further referred to as AI-human visual alignment. Specifically, we propose a new dataset for measuring AI-human visual alignment in terms of image classification, a fundamental task in machine perception. In order to evaluate AI-human visual alignment, a dataset should encompass samples with various scenarios that may arise in the real world and have gold human perception labels. Our dataset consists of three groups of samples, namely Must-Act (i.e., Must-Classify), Must-Abstain, and Uncertain, based on the quantity and clarity of visual information in an image and further divided into eight categories. All samples have a gold human perception label; even Uncertain (severely blurry) sample labels were obtained via crowd-sourcing. The validity of our dataset is verified by sampling theory, statistical theories related to survey design, and experts in the related fields. Using our dataset, we analyze the visual alignment and reliability of five popular visual perception models and seven abstention methods. Our code and data is available at https://github.com/jiyounglee-0523/VisAlign.
BibTeX
@misc{lee2023visalign,
  author = {Jiyoung Lee and Seungho Kim and Seunghyun Won and Joonseok Lee and Marzyeh Ghassemi and James Thorne and Jaeseok Choi and O.-Kil Kwon and Edward Choi},
  title = {{VisAlign: Dataset for Measuring the Degree of Alignment between AI and Humans in Visual Perception}},
  year = {2023},
  eprint = {2308.01525},
  archivePrefix = {arXiv},
  url = {https://arxiv.org/abs/2308.01525},
}
Figure