Thinking Fast and Slow in AI

Grady Booch, Francesco Fabiano, Lior Horesh, Kiran Kate, Jonathan Lenchner, Nick Linck, Andrea Loreggia, Keerthiram Murugesan, Nicholas Mattei, Francesca Rossi, Biplav Srivastava

Abstract: This paper proposes a research direction to advance AI which draws inspiration from cognitive theories of human decision making. The premise is that if we gain insights about the causes of some human capabilities that are still lacking in AI (for instance, adaptability, generalizability, common sense, and causal reasoning), we may obtain similar capabilities in an AI system by embedding these causal components. We hope that the high-level description of our vision included in this paper, as well as the several research questions that we propose to consider, can stimulate the AI research community to define, try and evaluate new methodologies, frameworks, and evaluation metrics, in the spirit of achieving a better understanding of both human and machine intelligence.

Introduction: The Limits of Narrow AI

Despite the meteoric rise of Deep Learning, modern AI remains fundamentally "narrow." While current models excel at specific tasks such as image classification or large-scale language modeling, they lack the robustness, generalizability, and common sense inherent in human intelligence.

The core challenge lies in a fundamental architectural gap. Drawing on Daniel Kahneman’s dual-process theory, current AI is largely a "System 1" machine—it is excellent at unconscious, fast, and data-driven pattern matching. However, it struggles with "System 2" capabilities: logical reasoning, long-term planning, and ethical deliberation. To move toward more holistic intelligence, we must understand how these two systems can interact, govern one another, and share knowledge.

A Conceptual Framework for Neuro-Symbolic AI

This position paper proposes a roadmap for the next generation of AI by mapping cognitive psychology onto architectural requirements. This framework, often referred to as Neuro-symbolic AI, seeks to integrate the strengths of both connectionist and symbolic approaches.

Mapping Cognitive Theory to Architecture

  • System 1 (The Intuitive): Maps to modern Machine Learning (ML) and Deep Learning. It is fast, parallel, and handles pattern recognition through statistical correlation.
  • System 2 (The Deliberative): Maps to symbolic logic, search, and optimization. It is slow, sequential, and handles explicit rule-following and causal reasoning.

Interaction Patterns and Knowledge Compilation

A critical aspect of this framework is how the two systems interact. System 2 can provide heuristics to narrow the search space for System 1, while System 2 can "train" System 1 over time. This mimics human expertise; for example, learning to drive begins as a slow, conscious System 2 effort but eventually "compiles" into an automatic System 1 habit.

Furthermore, the framework proposes a "governance" layer. This layer acts as an introspective monitor, deciding when the AI should switch from fast, low-cost intuition to slow, high-cost deliberation—a necessity in high-stakes or ethically complex scenarios.

Key Insights: Reasoning, Ethics, and Consciousness

The Sequentiality Debate

In humans, System 2 is sequential; we can typically only perform one complex reasoning task at a time. The authors pose a provocative question: Is this a limitation of biological hardware, or a "feature" of reasoning? It is possible that the bottleneck of sequential reasoning actually aids in the creation of useful abstractions, suggesting that AI System 2 architectures might benefit from similar constraints.

Ethics as System 2 Reasoning

Ethical decision-making often requires resolving "divergence"—conflicts between immediate, reward-seeking impulses (System 1) and long-term societal norms or logical constraints (System 2). The authors suggest that System 2 is the natural home for normative ethical theories, providing a logical check on the statistical biases of System 1.

Consciousness as a Tool

Drawing on the work of Graziano and Bengio, the paper suggests that a form of "M-consciousness" (a model of the mind) is necessary for an AI to understand its own competencies and limitations. This self-awareness allows the system to autonomously decide which internal "system" is best suited for a given challenge.

A Research Agenda for the AI Community

The primary contribution of this work is a structured research agenda consisting of 10 fundamental Research Questions, including:

  1. How do we define metrics for a hybrid System 1/System 2 AI?
  2. How do we model the "switching" mechanism (governance) between systems?
  3. How can we leverage this dual-system approach to handle ethical dilemmas and competing priorities?
  4. What architectural choices, such as multi-agent systems, best support this vision?

Conclusion

While this vision is high-level and does not provide a specific implementation, it serves as a call to action. By moving beyond pure connectionism or pure symbolism and focusing on the "integration route," the AI community can begin to develop systems that not only recognize patterns but truly understand the world through causal reasoning and common sense.

Summary updated

Abstract

This paper proposes a research direction to advance AI which draws inspiration from cognitive theories of human decision making. The premise is that if we gain insights about the causes of some human capabilities that are still lacking in AI (for instance, adaptability, generalizability, common sense, and causal reasoning), we may obtain similar capabilities in an AI system by embedding these causal components. We hope that the high-level description of our vision included in this paper, as well as the several research questions that we propose to consider, can stimulate the AI research community to define, try and evaluate new methodologies, frameworks, and evaluation metrics, in the spirit of achieving a better understanding of both human and machine intelligence.
BibTeX
@inproceedings{booch2021thinking,
  author = {Grady Booch and Francesco Fabiano and Lior Horesh and Kiran Kate and Jonathan Lenchner and Nick Linck and Andrea Loreggia and Keerthiram Murugesan and Nicholas Mattei and Francesca Rossi and Biplav Srivastava},
  title = {{Thinking Fast and Slow in AI}},
  booktitle = {AAAI Conference on Artificial Intelligence},
  year = {2021},
  url = {https://openreview.net/forum?id=5dFj0P42-n},
}
Figure