Viewpoint: When Will AI Exceed Human Performance? Evidence from AI Experts

Katja Grace, John Salvatier, Allan Dafoe, Baobao Zhang, Owain Evans

The question of when artificial intelligence will match or surpass human capability is a critical variable for global policy, economics, and safety. In the paper "When Will AI Exceed Human Performance? Evidence from AI Experts," Katja Grace and her colleagues from the Future of Humanity Institute and Yale University provide a comprehensive empirical look at what the world’s leading researchers believe about the future of their field.

Methodology and Data

The researchers surveyed 1,634 authors who published at the 2015 NIPS (now NeurIPS) and ICML conferences, receiving 352 responses. The study focused on "High-Level Machine Intelligence" (HLMI), defined as the point when unaided machines can accomplish every task better and more cheaply than human workers.

To ensure robust data, the authors used probabilistic elicitation, asking respondents for the years in which they believed there was a 10%, 50%, and 90% probability of HLMI occurring. They also tested for framing effects and used statistical modeling (Gamma CDFs) to generate aggregate forecasts.

The Regional Optimism Gap

One of the most striking findings was the significant difference in timelines predicted by researchers in different regions. Asian respondents were notably more optimistic, predicting that HLMI would arrive in just 30 years, whereas North American respondents predicted a much longer timeline of 74 years.

Regional differences in HLMI forecasts showing higher optimism in Asia compared to North America and Europe.

This gap suggests that cultural factors or local research environments significantly influence expert intuition regarding the pace of AI development.

Milestone Timelines and Full Automation

While the aggregate forecast predicts a 50% chance of HLMI by 2061 (45 years from the 2016 survey), specific milestones are expected much sooner. The experts predicted that AI would outperform humans in language translation by 2024 and truck driving by 2027. However, more complex tasks like surgery (2053) and mathematical research (2059) are expected to take decades longer.

Estimated timelines for specific AI milestones, ranging from game-playing and translation to full automation of labor.

Interestingly, the experts distinguished between HLMI (capability) and the Full Automation of Labor (the actual replacement of all human jobs). The median prediction for the latter was significantly further out, at 122 years from 2016.

Safety and Takeoff Dynamics

The survey also touched on the "intelligence explosion"—the idea that AI might rapidly become vastly superior to humans shortly after reaching HLMI. Most researchers viewed a rapid "takeoff" (within two years) as unlikely, assigning it a median probability of only 10%.

Regarding safety, the consensus was clear: 48% of respondents believe that research into minimizing the potential risks of AI should be prioritized more than it currently is, highlighting a growing concern within the technical community about the long-term implications of their work.

Limitations

The authors acknowledge that expert intuition is not a perfect crystal ball. Experts are susceptible to cognitive biases, framing effects, and the influence of current funding cycles. Despite these limitations, this study provides a vital "wisdom of the crowd" baseline for anticipating the social and economic disruptions AI may bring.

Summary updated

Abstract

Advances in artificial intelligence (AI) will transform modern life by reshaping transportation, health, science, finance, and the military. To adapt public policy, we need to better anticipate these advances. Here we report the results from a large survey of machine learning researchers on their beliefs about progress in AI. Researchers predict AI will outperform humans in many activities in the next ten years, such as translating languages (by 2024), writing high-school essays (by 2026), driving a truck (by 2027), working in retail (by 2031), writing a bestselling book (by 2049), and working as a surgeon (by 2053). Researchers believe there is a 50% chance of AI outperforming humans in all tasks in 45 years and of automating all human jobs in 120 years, with Asian respondents expecting these dates much sooner than North Americans. These results will inform discussion amongst researchers and policymakers about anticipating and managing trends in AI. This article is part of the special track on AI and Society.
BibTeX
@article{grace2018viewpoint,
  author = {Katja Grace and John Salvatier and Allan Dafoe and Baobao Zhang and Owain Evans},
  title = {{Viewpoint: When Will AI Exceed Human Performance? Evidence from AI Experts}},
  journal = {J. Artif. Intell. Res.},
  year = {2018},
  doi = {10.1613/jair.1.11222},
}
Figure