Abstract
To adapt to a changing world, we must be able to switch between rules already learned and, at other times, learn rules anew. Often we must do both at the same time, switching between known rules while also constantly re-estimating them. Here, we show these two processes, rule switching and rule learning, rely on distinct but intertwined computations, namely fast inference and slower incremental learning. To this end, we studied how monkeys switched between three rules. Each rule was compositional, requiring the animal to discriminate one of two features of a stimulus and then respond with an associated eye movement along one of two different response axes. By modeling behavior we found the animals learned the axis of response using fast inference (rule switching) while continuously re-estimating the stimulus-response associations within an axis (rule learning). Our results shed light on the computational interactions between rule switching and rule learning, and make testable neural predictions for these interactions.
Introduction: The Balance of Inference and Learning
Cognitive flexibility requires balancing two distinct learning mechanisms. Incremental learning, often modeled via reinforcement learning, is a slow, error-driven process used to build new habits or map novel stimuli to actions. In contrast, inferential switching (or Bayesian inference) allows for rapid jumps between known contexts or rules based on minimal evidence.
In complex, real-world environments, these processes often occur simultaneously. We may identify a general context (e.g., "I am driving in a city") but still need to refine specific stimulus-response mappings. This study investigates whether the brain treats rules as monolithic blocks or employs a multi-layered approach that combines fast context-switching with continuous, slow refinement of rule details.
Methods and Task Design
The researchers trained two rhesus macaques (Monkey S and Monkey C) on a perceptual categorization task involving three compositional rules. Each rule required the monkey to attend to a specific feature (Shape or Color) and respond along a specific axis (Axis 1 or Axis 2).
As shown in the figure above, the rules were structured as follows:
- Rule 1: Categorize Shape (Bunny vs. Tee) $\rightarrow$ Respond on Axis 1 (Upper-Left/Lower-Right).
- Rule 2: Categorize Color (Red vs. Green) $\rightarrow$ Respond on Axis 2 (Upper-Right/Lower-Left).
- Rule 3: Categorize Color (Red vs. Green) $\rightarrow$ Respond on Axis 1 (Upper-Left/Lower-Right).
Monkeys were not explicitly cued; they had to infer the active rule through trial-and-error feedback. Stimuli were 3D-rendered "morphed" objects, allowing for precise control over categorization difficulty. To explain the behavior, three models were compared: an Incremental Learner (QL), an Ideal Observer (IO), and a Hybrid Q-Learner (HQL) which uses Bayesian inference for axis selection and incremental learning for feature mapping.
Key Results: Fast Inference vs. Slow Learning
The behavioral analysis revealed a striking dissociation between axis discovery and feature mapping.
Fast Axis Switching
Monkeys identified the correct response axis almost immediately after a block switch. For instance, Monkey S responded on the correct axis within 5 trials in 97% of blocks. This rapid adaptation suggests a fast inferential process rather than slow incremental learning for the general "context" of the response.
The "Two-Latent State" Strategy and Slow Feature Learning
Despite fast axis switching, learning the specific feature mapping (e.g., distinguishing whether to use color or shape on Axis 1) was significantly slower. The monkeys appeared to represent only two latent states corresponding to the two response axes, rather than three distinct rules.
- Axis 2 ("Clean"): Since only Rule 2 utilized Axis 2, performance reached asymptotic levels almost immediately upon switching to this axis.
- Axis 1 ("Messy"): Because Rules 1 and 3 shared Axis 1 but required different feature categorizations, the monkeys had to "relearn" the mapping every time they switched back to Axis 1.
The "smoking gun" for this incremental learning was the incongruency effect. On Axis 1, monkeys performed significantly better on "congruent" trials (where Rule 1 and Rule 3 required the same response) than on "incongruent" trials. A pure inferential model (Ideal Observer) would have identified the rule and performed equally well on both, but the monkeys' behavior showed a performance gap of approximately 31%, perfectly captured by the Hybrid Q-Learner (HQL) model.
Discussion and Neural Implications
The study demonstrates that monkeys use a hybrid strategy: fast Bayesian inference to identify the response axis (context) and slow reinforcement learning to map features to actions within that context. By maintaining separate latent states for each axis, the monkeys partially avoided "catastrophic interference," where learning on one axis would overwrite knowledge of the other.
These findings lead to a testable neural prediction: the brain likely dissociates these processes. The prefrontal cortex may handle the fast inferential belief about the current state (the axis), while the basal ganglia manages the slower incremental learning of associations within that state. Future research could explore whether different training protocols could force the brain to split "messy" axes into distinct, stable rule representations.
Summary updated