Measuring the Occupational Impact of AI: Tasks, Cognitive Abilities and AI Benchmarks (Extended Abstract)

Songül Tolan, Annarosa Pesole, Fernando Martínez-Plumed, Enrique Fernández-Macías, José Hernández-Orallo, Emilia Gómez

Introduction

While the transformative potential of Artificial Intelligence (AI) on the labor market is widely acknowledged, quantifying its specific impact remains a significant challenge. Traditional economic studies often focus on physical automation ("robotization") or general digitalization, but AI is unique because it performs cognitive tasks that lack a direct one-to-one mapping to specific job titles. This research introduces a novel framework to bridge the gap between AI research benchmarks and human work tasks by using a common language of 14 cognitive abilities.

Methodology: A Three-Layer Mapping Framework

The researchers developed a multi-layered correspondence model to translate AI research intensity into occupational exposure. The framework consists of three primary layers:

  1. Occupational Tasks: 59 generic work tasks extracted from major labor databases (EWCS, PIAAC, and O*Net) across 119 standardized occupations.
  2. Cognitive Abilities: An intermediate layer of 14 abilities (e.g., Visual Processing, Logical Reasoning, Social Interaction) derived from cognitive science.
  3. AI Benchmarks: A comprehensive list of 328 benchmarks used to evaluate AI progress (e.g., ImageNet, RoboCup, Turing Test).
The three-layer mapping framework connecting occupations to AI research benchmarks via cognitive abilities.

The "bridge" between these layers was constructed using the Delphi Method, where expert consensus created correspondence matrices. By calculating the "research intensity" (volume of publications and news) for each benchmark, the authors could flow that intensity through the cognitive layer to determine which occupations are most exposed to current AI trends.

The study categorizes cognitive abilities into three domains: People, Ideas, and Things. The findings reveal that AI research is heavily concentrated on "Ideas" (memorization, planning, learning) and "People" (communication), while "Things" (navigation and physical interaction in complex environments) receive comparatively less research intensity.

Interestingly, despite the prominence of Large Language Models, the study identified a "Social Interaction Gap." There is currently near-zero research intensity in "Mind modelling and social interaction" (MS) and "Metacognition" (MC), suggesting that AI still struggles with the nuanced social contexts that define many human-centric roles. Furthermore, basic human skills like "naive physics" are often missing from benchmarks because they are intuitively "easy" for humans, creating a blind spot in AI evaluation.

Occupational Exposure and Economic Implications

Unlike previous waves of automation that primarily targeted low-wage manual labor, this framework suggests that AI exposure is highest in high-skill, high-wage roles.

  • High-Skill Exposure: General office clerks, medical doctors, and teachers show the highest exposure to current AI research intensity.
  • Low-Skill Resilience: Occupations requiring high physical or environmental interaction—such as cleaners, waiters, and shop salespersons—are currently less exposed.
  • The Wage Correlation: There is a clear positive relationship between wage percentiles and AI exposure. High-income occupations are significantly more likely to be affected by current AI research trends than low-income ones.
Correlation between wage percentiles and AI exposure scores across various occupations.

Discussion and Future Directions

It is critical to note that the framework measures exposure, not necessarily displacement. High exposure could indicate that a job is ripe for automation, but it could also mean that AI will significantly augment the worker's capabilities.

Limitations of the study include potential benchmark bias—where abilities ignored by the AI community appear "safe"—and the static nature of the Delphi-based mapping. Future work will focus on how these mappings evolve as AI techniques transition from symbolic systems to deep learning. This framework provides a vital tool for policymakers to direct education and retraining efforts toward abilities that remain uniquely human or are less targeted by current AI research.

Summary updated

Abstract

We present a framework for analysing the impact of AI on occupations. This framework maps 59 generic tasks from different occupational datasets to 14 cognitive abilities and these to a comprehensive list of 328 AI benchmarks used to evaluate research intensity in AI. The use of cognitive abilities as an intermediate layer allows for an identification of potential AI exposure for tasks for which AI applications have not been explicitly programmed. We provide insights into the abilities through which AI is most likely to affect jobs, and we show how some of the abilities where AI research is currently very intense are linked to tasks with comparatively limited labour input in the labour markets of advanced economies.
BibTeX
@inproceedings{tolan2022measuring,
  author = {Songül Tolan and Annarosa Pesole and Fernando Martínez-Plumed and Enrique Fernández-Macías and José Hernández-Orallo and Emilia Gómez},
  title = {{Measuring the Occupational Impact of AI: Tasks, Cognitive Abilities and AI Benchmarks (Extended Abstract)}},
  booktitle = {International Joint Conference on Artificial Intelligence},
  year = {2022},
  doi = {10.24963/ijcai.2022/809},
}
Figure