An organism navigating a complex environment faces a fundamental computational dilemma. To maximize rewards, it can deploy a goal-directed (or model-based) strategy, simulating the future consequences of its actions step-by-step through a mental map of the world. This strategy is highly flexible but computationally expensive and slow. Alternatively, it can rely on a habitual (or model-free) strategy, executing cached stimulus-response associations. This approach is incredibly fast and computationally cheap, but it is completely blind to sudden changes in outcome value.
As reviewed by Raymond J. Dolan et al., understanding how the brain manages these two primary modes of control is a foundational challenge in cognitive science. Rather than operating as a strict dichotomy, these systems exist on a spectrum, constantly interacting, competing, and cooperating to optimize behavior. The core problem the brain must solve is arbitration: determining which system should take precedence at any given moment to maximize reward while minimizing cognitive effort.
The Logic of the Switch: Speed, Accuracy, and Uncertainty
Why not always plan, or always rely on habit? The brain must dynamically decide when to invest heavy cognitive resources and when to hand control over to autopilot.
Mohammad Mehdi Keramati et al. formalized this choice as a normative speed/accuracy trade-off. When a decision requires high precision and the environment is unfamiliar, the goal-directed system takes charge. As actions are repeated and their outcomes become predictable, the rapid habitual system takes over to save time and energy.
This arbitration is deeply sensitive to environmental statistics. Dylan A. Simon et al. demonstrated that humans strategically shift between planning and habit based on the level of noise and volatility in their surroundings. When the environment is highly uncertain or changing rapidly, relying on stale, cached habits is dangerous, forcing a shift back to deliberate planning.
However, full tree-search planning quickly runs into "computational ruin" in complex environments. To mitigate this, the brain employs hard-wired shortcuts to simplify the math. For instance, Neir Eshel et al. showed that a Pavlovian mechanism "prunes" decision trees, reflexively halting the evaluation of future paths as soon as a major potential loss is encountered.
Blurring the偏Dichotomy: Continua and Intermediate Representations
While treating habits and goals as two separate boxes is conceptually useful, recent theoretical work suggests a more integrated architecture.
One prominent bridge is the Successor Representation (SR). As discussed in The Successor Representation: Its Computational Logic and Neural Substrates, the SR bypasses full step-by-step planning by encoding the environment as a predictive map of future state occupancies. This allows the brain to retain the flexibility of goal-directed planning (adapting instantly to changes in reward values) while matching the computational speed of habits.
From a structural perspective, Paul F. Kinghorn et al. proposed a Hierarchical Predictive Coding framework where habits and deliberate actions are not separate systems at all, but rather different levels of processing within a single neural architecture. When lower layers have sufficient information, they drive rapid, habitual actions; when uncertainty rises, higher layers are recruited for reflective control.
This dynamic coordination can be observed empirically through reaction times. By modeling both choices and decision latencies, Guillaume Viejo et al. and Guillaume Viejo et al. showed that the brain does not simply choose one system over the other; instead, a continuous interplay between adaptive working memory and reinforcement learning systems best explains human behavioral data.
How Motivation Energizes and Directs Behavior
Motivation plays a dual role in this arbitration process, acting differently on each system. Peter Dayan proposed a tripartite model separating goal-directed action selection from habitual vigor, highlighting that motivation includes a general arousal component distinct from specific action selection.
Expanding on this, Yael Niv et al. introduced a normative framework distinguishing between the "directing" and "energizing" effects of motivation:
- Goal-directed actions are guided by outcome-specific directing effects. They are sensitive to specific outcome utilities (e.g., planning a route to a specific restaurant because you want pizza).
- Habits, conversely, are functionally blind to specific outcomes at the moment of decision-making. This property was neurally verified by Mimi Liljeholm et al., who showed that habit-related brain structures encode motor responses but lack outcome representations. Instead, habits are influenced by an energizing effect akin to generalized drive, which increases response vigor or speed irrespective of the specific outcome value.
Neural Circuitry and Clinical Imbalances
The physical arbitration between these systems relies on coordinated prefrontal-striatal networks. For example, Matthew Baker identified a specific circuit from the external globus pallidus (GPe) to the dorsolateral striatum (DLS) that actively constrains habit formation to preserve goal-directed control. When behavioral strategies change completely—such as during the extinction of a reward-seeking behavior—Eleonora Russo et al. demonstrated that the medial prefrontal cortex undergoes a population-wide transition into a new stable attractor state, rather than a gradual unlearning of the habit.
When this delicate arbitration machinery breaks down, psychiatric symptoms can emerge. Obsessive-Compulsive Disorder (OCD) is increasingly understood as a pathological tilt toward habitual control. Taekwan Kim et al. showed that individuals with OCD exhibit a habitual bias driven by heightened uncertainty within their goal-directed system, a deficit mapped to weakened connectivity between the prefrontal cortex and the putamen. Furthermore, Valerie Voon et al. discovered that this imbalance is highly dependent on motivational valence: individuals with OCD lean heavily toward habits when anticipating rewards, but revert to goal-directed control when facing potential losses.
Intriguingly, this computational split is so fundamental to human behavior that it shapes our social cognition. Tobias Gerstenberg et al. demonstrated that everyday human intuition includes a "theory of habits." We naturally use cues like decision speed and repetition to infer whether others are acting deliberately or on autopilot, adjusting our moral judgments of their actions accordingly.
Key Foundational Work
A normative perspective on motivation (2006) — contributed 9 concept elements to this direction.
Motivated Reinforcement Learning (2001) — contributed 2 concept elements to this direction.