On Benchmarking Human-Like Intelligence in Machines

Lance Ying, Katherine M. Collins, Lionel Wong, Ilia Sucholutsky, Ryan Liu, Adrian Weller, Tianmin Shu, Thomas L. Griffiths, Joshua B. Tenenbaum

Abstract

Recent benchmark studies have claimed that AI has approached or even surpassed human-level performances on various cognitive tasks. However, this position paper argues that current AI evaluation paradigms are insufficient for assessing human-like cognitive capabilities. We identify a set of key shortcomings: a lack of human-validated labels, inadequate representation of human response variability and uncertainty, and reliance on simplified and ecologically-invalid tasks. We support our claims by conducting a human evaluation study on ten existing AI benchmarks, suggesting significant biases and flaws in task and label designs. To address these limitations, we propose five concrete recommendations for developing future benchmarks that will enable more rigorous and meaningful evaluations of human-like cognitive capacities in AI with various implications for such AI applications.

The Crisis of Benchmark Saturation

We are currently in an era of "benchmark saturation," where AI models frequently claim to reach or surpass "human-level" performance on complex cognitive tasks. However, these claims are often hollow because of a fundamental misalignment between how AI is tested and how humans actually think.

Most benchmarks rely on "hard labels"—a single correct answer—for tasks that are inherently subjective, ambiguous, or context-dependent, such as moral reasoning or irony detection. Furthermore, many benchmarks are "cognitively inspired" but lack actual human data to validate their labels. If humans do not agree on the "correct" answer, an AI matching a designer-provided label cannot truly be called "human-like."

Methodology: Capturing the Human Distribution

To stress-test the validity of existing benchmarks, a human evaluation study was conducted involving 240 participants recruited via Prolific. The study sampled 300 stimuli from 10 well-known AI benchmarks across three major suites.

Summary of benchmarks and tasks evaluated in the study.

Unlike traditional forced-choice evaluations, this study employed "soft labeling." Participants used a slider (1–100) to indicate their level of agreement or the likelihood of an option. This approach allowed researchers to capture the full distribution of human uncertainty and disagreement, comparing these distributions against the original "ground truth" labels.

Results: The Disconnect Between Benchmarks and Human Intuition

The comparative analysis revealed a significant gap between benchmark standards and actual human judgment. On average, only 63.51% of human participants agreed with the "ground truth" labels provided by the benchmarks.

Distribution of human agreement rates across stimuli, showing that many benchmark labels represent minority opinions.

Key findings include:

  • The Majority Fallacy: For 26.67% of the stimuli tested, less than half of the human participants selected the label provided by the benchmark. In these cases, the "correct" answer according to the benchmark was actually a minority opinion among humans.
  • High Variability in Social Tasks: Tasks like "Social Support" showed extremely low agreement (32.13%), indicating that benchmark labels were often disconnected from human intuition.
  • Gradedness of Judgment: Human judgment is inherently graded; 57.69% of ratings fell between 20 and 80 on the slider. AI models, however, are typically rewarded for high-confidence binary predictions that ignore this natural uncertainty.

Toward Pluralistic Alignment

The study highlights that disagreement is often a feature of human cognition, not a bug. In tasks involving perception or social navigation, human disagreement reveals underlying cognitive mechanisms. Benchmarks that filter out "ambiguous" items to increase inter-annotator agreement effectively remove the most cognitively interesting data.

To move forward, the authors recommend "Pluralistic Alignment." Evaluation should shift toward comparing model output distributions to human population distributions using metrics like KL-divergence or Wasserstein distance. This ensures that AI models are evaluated on their ability to represent the rich, probabilistic nature of real human intelligence rather than arbitrary, singular standards.

Conclusion and Future Work

While collecting large-scale human distribution data is resource-intensive, it is necessary for meaningful AI evaluation. Future work should focus on developing scalable ways to collect soft labels and creating "ecologically rich" tasks that require multi-modal reasoning and long-term interaction, moving beyond static text snippets to more realistic social simulations.

Summary updated

Abstract

Recent benchmark studies have claimed that AI has approached or even surpassed human-level performances on various cognitive tasks. However, this position paper argues that current AI evaluation paradigms are insufficient for assessing human-like cognitive capabilities. We identify a set of key shortcomings: a lack of human-validated labels, inadequate representation of human response variability and uncertainty, and reliance on simplified and ecologically-invalid tasks. We support our claims by conducting a human evaluation study on ten existing AI benchmarks, suggesting significant biases and flaws in task and label designs. To address these limitations, we propose five concrete recommendations for developing future benchmarks that will enable more rigorous and meaningful evaluations of human-like cognitive capacities in AI with various implications for such AI applications.
BibTeX
@misc{ying2025benchmarking,
  author = {Lance Ying and Katherine M. Collins and Lionel Wong and Ilia Sucholutsky and Ryan Liu and Adrian Weller and Tianmin Shu and Thomas L. Griffiths and Joshua B. Tenenbaum},
  title = {{On Benchmarking Human-Like Intelligence in Machines}},
  year = {2025},
  eprint = {2502.20502},
  archivePrefix = {arXiv},
  url = {https://arxiv.org/abs/2502.20502},
}

Loading related content...

Figure