TL;DR
This paper argues that current AI successes in machine learning, particularly deep learning, are akin to "System 1" thinking – fast but opaque. To achieve robust and general intelligence, AI needs to integrate these "System 1" learners with "System 2" model-based solvers, which are slow but transparent and analytical. The authors review the history and current state of both approaches, highlighting the challenges and potential of their integration.
What Problem Does This Paper Address?
The paper addresses the long-standing challenge in Artificial Intelligence of achieving general intelligence. While recent breakthroughs in machine learning, especially deep learning and deep reinforcement learning, have led to impressive performance in specific tasks like games, these "model-free learners" are often black boxes. They lack the flexibility, transparency, and generality of "model-based solvers," which can tackle well-defined problems but require explicit models and can be computationally expensive. The core problem is bridging the gap between these two paradigms to create AI systems that are both capable and understandable, mirroring the human mind's dual-process system (System 1 and System 2).
Methods
The paper doesn't introduce new algorithms but rather provides a conceptual framework and historical review. It contrasts two main approaches:
- Model-Free Learners: These systems, exemplified by deep learning (DL) and deep reinforcement learning (DRL), learn a function (e.g., a policy or classifier) directly from data or experience. They use algorithms like stochastic gradient descent to adjust parameters of fixed structures (like neural networks). Their strength lies in learning complex patterns from vast amounts of data, but they are often opaque and struggle with generalization to unseen problem structures.
- Model-Based Solvers: These programs take a description of a model instance (e.g., a planning problem, a SAT instance) and compute a solution. They work "out of the box" without training, deriving solutions lazily for specific inputs. Examples include classical planners, SAT solvers, and Bayesian network solvers. Their strength lies in their generality and transparency, as they operate based on explicit models. However, they require well-defined models and can face scalability issues with complex problems.
The paper also discusses specific techniques within planning, a type of solver, such as:
- Relaxations: Simplifying a problem to make it easier to solve, with the solution providing a heuristic for the original problem.
- Transformations: Converting problems into a form that can be solved by existing solvers (e.g., classical planners).
- Width: A search algorithm (IW(k)) that exploits problem structure to achieve efficient planning, even for complex instances.
Datasets
Not discussed in the paper. The paper focuses on conceptual comparisons and theoretical challenges rather than empirical evaluation on specific datasets.
Key Insights and Ideas
- System 1 vs. System 2 Analogy: The paper draws a powerful analogy between model-free learners and System 1 (fast, intuitive, opaque) and model-based solvers and System 2 (slow, analytical, transparent) of human cognition. This framing helps explain the strengths and weaknesses of each AI approach.
- The Need for Integration: Current AI successes are largely System 1-like. To achieve general intelligence, AI needs to integrate these fast learners with the deliberate, analytical capabilities of System 2-like solvers.
- Solvers as Foundation for Generality: While learners excel at pattern recognition, solvers provide the framework for robust reasoning and generalization, especially when dealing with novel problems or requiring explanations.
- Generalized Planning as a Bridge: The concept of generalized planning, where a single plan can solve a family of problems, represents a promising area where learners and solvers can meet.
- Limitations of Current Learners: The fixed input size of neural networks is a significant limitation for learners, preventing them from emulating solvers that can handle arbitrary problem instances.
Key Results
The paper does not present quantitative results from experiments. Instead, its key "results" are conceptual:
- A clear articulation of the differences and parallels between model-free learners and model-based solvers.
- An argument for the necessity of integrating these two paradigms for achieving general AI.
- An overview of planning techniques (relaxations, transformations, width) that highlight the power and scalability of model-based approaches.
- The identification of key challenges for future AI research, including model learning, learning relevant variables, and learning abstract representations.
Limitations and Future Work
Limitations:
- Input Size Restriction: A major limitation of current model-free learners (especially deep learning) is their fixed input size, which prevents them from handling arbitrarily sized problems that solvers can manage.
- Scalability of Solvers: While solvers are general, their computational complexity can be a bottleneck for very large or complex problems.
Future Work:
- Model Learning: Developing algorithms that can learn models from streams of actions and partial observations is crucial.
- Learning Relevant Variables: Identifying and learning the key variables that define a problem is essential for both learners and solvers.
- Learning Abstract Representations: Creating fixed-size, goal-dependent abstractions of problems can help overcome the input size limitations of learners.
- Integration of Systems 1 and 2: The paper strongly advocates for research into synergistic integrations of learners and solvers, drawing inspiration from human cognition. This includes dealing with other agents and the complexities of the physical and social world.
- Addressing "Dreams and Nightmares": The paper concludes by emphasizing the ethical implications of AI and the need for flexible, transparent systems that can be guided by human values, suggesting that a well-functioning "System 2" is vital for responsible AI development.
Summary updated