Introduction
Most modern theories of learning, spanning from neuroscience to artificial intelligence, rely heavily on "feedback logic." In this paradigm, an organism evaluates its current state, calculates an error signal—such as a reward prediction error—and uses that signal to modify its behavior. However, a significant theoretical gap remains: how do evolutionary values, which operate over generational timescales, translate into the immediate, second-by-second learning of an individual?
This paper proposes a selectionist alternative where the generation of new behaviors is independent of evaluation criteria, mirroring the "blind" nature of biological evolution. By applying evolutionary principles of variation and selection to individual learning, the author suggests that adaptation is driven by innate behavioral trajectories that act as attractors.
The Automaton Framework
To formalize this theory, the author utilizes an automata theory approach to define an adaptive agent. The agent is represented as an automaton ($A$) composed of distinct sets of components that interact with one another and the environment.
In this model:
- The Automaton ($A$): Defined by components representing the environment ($C_E$), the body ($C_I$), internal brain states ($C_S$), and actions ($C_A$).
- Behavior ($B$): The transformation function dictating how these states evolve over time.
- Learning ($L$): A higher-order transformation that modifies the behavior function $B$.
Contrasting Feedback and Evolutionary Logic
The core of the paper lies in the mathematical distinction between two types of learning logic. In feedback logic, the learning transformation $L$ is a direct function of the current state; the modification is determined by the evaluation of the situation. In contrast, evolutionary logic treats $L$ as a random process (noise or mutation) applied to behavior $B$. While the magnitude of this variation may be state-dependent, the direction of the variation remains blind.
Under evolutionary logic, the organism produces a variety of behaviors first, and selection occurs afterward. This "generation then selection" sequence is the inverse of standard reinforcement learning models.
Learning as Trajectory Recovery
The author posits that innate behaviors, shaped by natural selection, form a "primary repertoire" of trajectories in phase space. These trajectories represent "safe" or "approved" paths for the organism. Learning is triggered specifically when an unexpected environmental reaction pushes the agent's state away from its target trajectory.
When this deviation occurs, the agent enters a state of high "blind variation." The internal target trajectory acts as a "vicarious selector"—an internal stand-in for natural selection. If a random variation successfully leads the agent back to the target attractor, that new sequence is retained and added to the agent's repertoire. Unlike earlier homeostatic models, such as those proposed by W. Ross Ashby, this framework emphasizes knowledge retention: new learned branches are integrated into the attractor without altering existing adaptive habits.
Contributions and Future Directions
This theoretical work provides several key conceptual contributions:
- Formalization: It establishes a mathematical distinction between feedback-based and selection-based learning.
- Initiation Criteria: It defines the start of learning as a critical deviation from an established behavioral trajectory.
- Selection Criterion: It identifies the return to an "approved" trajectory as the primary mechanism for retaining new behaviors.
While the framework is formally rigorous, the author acknowledges it is currently insufficient for direct implementation in animal learning models or animat algorithms. Future work must identify the specific cellular and neuronal mechanisms—such as types of synaptic plasticity—that support this trajectory-based paradigm. Additionally, defining what constitutes a "deviation" versus a "novel but safe state" in complex, high-dimensional environments remains a significant challenge for future research.
Summary updated