A multidisciplinary task-based perspective for evaluating the impact of AI autonomy and generality on the future of work

Enrique Fernández‐Macías, Emília Gómez, José Hernández‐Orallo, Bao Sheng Loe, Bertin Martens, Fernando Martínez‐Plumed, Songül Tolan

TL;DR

This position paper argues that predicting AI’s impact on the future of work requires bridging a massive disciplinary divide between economics and computer science. By proposing a unified framework that maps socio-economic definitions of "tasks" to computational ones, the authors highlight that the true disruptive potential of modern AI lies in its rapidly increasing autonomy and generality—traits historically reserved for human workers.

What Problem Does This Paper Address?

Everyone wants to know which jobs AI will automate, but current research is hampered by narrow, siloed views of the problem. Economists and social scientists view "tasks" through a socio-economic lens (e.g., physical vs. intellectual, routine vs. non-routine, autonomous vs. standardized). Meanwhile, computer scientists view "tasks" as encapsulated input-output processes (e.g., classifying an image in a Kaggle dataset).

This misalignment creates a blind spot. Because computer science often artificially restricts tasks to highly specific benchmarks to measure progress, it fails to capture the messy, interactive nature of real-world occupations. Furthermore, current labor models often ignore human attributes like personality, cognitive adaptability, and group dynamics. The paper addresses this gap by proposing a multidisciplinary framework to evaluate how AI will alter the workplace.

Methods

As a position paper, this work does not present a traditional experimental methodology or a new algorithm. Instead, it introduces a conceptual framework for analyzing AI and labor:

  1. Target Specification: Clearly defining the labor market in terms of occupations (bundles of tasks), tasks (units of work), and skills (human capabilities influenced by cognitive ability and personality).
  2. Task Mapping: Contrasting and integrating the socio-economic perspective (which categorizes tasks by content and tools) with the computational perspective (which defines tasks via datasets, ground truths, and evaluation metrics).
  3. Impact Assessment: Evaluating the trajectory of AI progress specifically through the lenses of Autonomy (a system's ability to pursue unprogrammed subgoals and adapt) and Generality (the degree to which a system covers a wide range of tasks).

Datasets

Not discussed in the paper in the context of training or evaluating a novel model.

Note: The authors do discuss existing benchmarking platforms (like Kaggle and Mediaeval) and crowdsourcing platforms (like Amazon Mechanical Turk) as examples of how the computer science field currently defines and encapsulates tasks.

Key Insights and Ideas

  • The Definition of a "Task" is Fragmented: To an economist, a task might be "managing a team" (a social, non-routine task). To an AI researcher, a task is "predicting user preferences based on a dataset." Bridging this gap is essential to understanding automation.
  • The Benchmark Trap: The AI community's reliance on encapsulated, static benchmarks (like image recognition datasets) oversimplifies what work actually is. Real-world jobs are interactive processes with shifting goals, not just input-output mappings.
  • The Historical Paradox of Automation: Historically, scientific management (Taylorism) drove economic progress by reducing human autonomy and generality—breaking complex jobs into narrow, repetitive, routine tasks. Ironically, AI development is moving in the exact opposite direction, striving to create machines with more autonomy and generality.
  • Workers as "Cognitive Resources": As AI systems become capable of learning and adapting like humans, HR and management decisions will shift. The question will change from "Which human has the skills for this task?" to "Is a human or a machine the most suitable cognitive resource for this bundle of tasks?"

Key Results

Because this is a theoretical position paper, there are no quantitative results (e.g., accuracy, speedup, or specific job-loss percentages). The primary "result" is the proposed multidisciplinary framework itself, which provides a structured vocabulary and a roadmap for future researchers to map computational AI capabilities directly onto socio-economic labor models.

Limitations and Future Work

The authors acknowledge that evaluating highly general and autonomous AI systems presents a massive challenge. Current evaluation protocols work well for narrow tasks (e.g., a movie recommender), but fail when assessing broad "abilities" or "skills."

For future work, the authors suggest:

  • Developing new metrics to measure AI generality (e.g., does a system just have a large fixed menu of tasks, or can it learn entirely new ones?).
  • Shifting AI evaluation away from pure task performance (accuracy) toward dimensions that matter in the workplace: reliability over time, fairness, transparency, data efficiency, and accountability in high-stakes environments.

Summary updated

Abstract

This paper presents a multidisciplinary task approach for assessing the impact of artificial intelligence on the future of work. We provide definitions of a task from two main perspectives: socio-economic and computational. We propose to explore ways in which we can integrate or map these perspectives, and link them with the skills or capabilities required by them, for humans and AI systems. Finally, we argue that in order to understand the dynamics of tasks, we have to explore the relevance of autonomy and generality of AI systems for the automation or alteration of the workplace.
BibTeX
@inproceedings{fernandezmacias2018multidisciplinary,
  author = {Enrique Fernández‐Macías and Emília Gómez and José Hernández‐Orallo and Bao Sheng Loe and Bertin Martens and Fernando Martínez‐Plumed and Songül Tolan},
  title = {{A multidisciplinary task-based perspective for evaluating the impact of AI autonomy and generality on the future of work}},
  booktitle = {IJCAI},
  year = {2018},
}
Figure