Abstract
This paper proposes a unifying, sub-symbolic, computational theory of cognition that integrates psychological theories, artificial intelligence, and neuroscience. By leveraging the free energy principle, the framework explains how abstract cognitive knowledge is learned from actively gathered sensorimotor experiences. It posits that symbols are emergent neural attractors grounded in physical interactions, offering a quantitative bridge between low-level processing and high-level thought.
Introduction: Bridging the Cognitive Divide
For decades, cognitive science has grappled with a fundamental split. Embodied Cognition (EC) argues that thinking is deeply rooted in sensory experiences and physical bodies, while Classical AI relies on symbolic logic and production rules. The "missing link" is a formal computational bridge: how does a brain transition from raw sensorimotor data—like the weight of an object—to abstract concepts like "containment"?
Martin Butz addresses this by proposing a sub-symbolic theory where "symbols" are stable patterns of neural activity within a predictive, generative model. This framework synthesizes theories of event segmentation, anticipatory behavior, and predictive coding into a single, mathematically grounded architecture.
The Predictive Processing Loop
At the heart of this theory is a continuous loop of predictive processing. Borrowing from Karl Friston’s free energy principle, the brain is viewed as a generative model striving to minimize prediction error (free energy).
The loop functions through three fundamental types of predictive encodings:
- Spatial: Mapping frames of reference (e.g., hand position relative to gaze).
- Top-down: Generalizing features and "Gestalt" perceptions (e.g., object recognition).
- Temporal: Predicting how actions and forces cause changes over time.
Neural activity-oriented inference causes the continuous adaptation of these active encodings, where top-down predictions and bottom-up error signals interact to settle into a "global posterior" state. Meanwhile, neural structure-oriented inference leads to long-term adaptation of the generative model as a whole.
Grounded Semantics and Symbols as Attractors
A key insight of this framework is that "symbols" and "thoughts" are not hard-coded primitives but distributed neural attractors. When a system perceives or imagines a situation, such as a "ball in a bowl," it activates a set of predictive encodings that reach a local free-energy minimum through mutual consistency.
Abstract concepts are thus grounded in layers of simpler sensorimotor episodes. For instance, the concept of "in" (containment) is learned through physical experiences of objects being placed in containers. Imagination and abstract thought occur when the system "decouples" from actual sensory input, allowing the internal generative model to run freely to explore hypothetical scenarios or future states.
Core Propositions of the Theory
The theory is summarized by nine main propositions that define its scope:
- Modularity: The brain is a modular, probabilistic, predictive system.
- Encoding Types: Encodings are categorized as temporal, spatial, and top-down.
- Abstraction: Modularity enables effective hierarchical abstraction.
- Anticipation: Thought is inherently anticipatory and driven by homeostasis.
- Concepts as Minima: Concepts are consistent free-energy minima (attractors).
- Compositionality: Relating these minima allows for complex, compositional thought.
- Scenarios: Perceived situations are compositional concepts.
- Episodes: Continuous experience is segmented into concatenations of event schemata.
- Ideas: Thinking is the active exploration of these conceptual spaces.
Future Directions and Limitations
While providing a robust roadmap, the theory faces several challenges. A "fully integrative implementation" combining all proposed biases does not yet exist. Furthermore, training such an agent requires massive amounts of embodied data. The author suggests using high-fidelity physics simulators (like MuJoCo) or game engines as "digital wombs" for cognitive development.
Future work must also address the social and linguistic dimensions of cognition—specifically how social interaction shapes these neural attractors and how the system might encode the intentions and knowledge of other agents in the environment.
Summary updated