Abstract: Advances in artificial intelligence are stimulating interest in neuroscience. However, most attention is given to discrete tasks with simple action spaces, such as board games and classic video games. Less discussed in neuroscience are parallel advances in "synthetic motor control". While motor neuroscience has recently focused on optimization of single, simple movements, AI has progressed to the generation of rich, diverse motor behaviors across multiple tasks, at humanoid scale. It is becoming clear that specific, well-motivated hierarchical design elements repeatedly arise when engineering these flexible control systems. We review these core principles of hierarchical control, relate them to hierarchy in the nervous system, and highlight research themes that we anticipate will be critical in solving challenges at this disciplinary intersection.
Introduction
This perspective bridges the gap between biological motor control and the development of "synthetic" motor systems in AI. While neuroscience has traditionally focused on simple, optimized movements (like reaching for a button), and AI has focused on discrete tasks (like playing Chess), the future of both fields lies in hierarchical control: the art of breaking down complex, full-body behavior into reusable, autonomous layers.
In neuroscience, the dominant framework is Optimal Feedback Control (OFC), which treats movement as a mathematical optimization problem. While powerful, OFC struggles to explain how animals perform thousands of different skills or reuse parts of one movement (like a step) for another (like a jump). In AI, "flat" reinforcement learning (RL) often fails when applied to high-dimensional humanoid bodies because the search space is too vast. Hierarchical approaches move beyond these "toy" tasks toward robust, full-body intelligence.
Core Principles of Hierarchical Control
The authors propose a conceptual framework based on six principles distilled from recent AI successes and classical neuroscience:
- Information Factorization: Routing only relevant data to specific layers (e.g., the spine doesn't need a global map to maintain balance).
- Partial Autonomy: Lower levels (like the spine) can function without constant "micromanagement" from the top.
- Amortized Control: "Caching" successful movements so they can be triggered instantly without re-calculating physics.
- Modular Objectives: Different parts of the system are trained on different goals (e.g., balance vs. navigation).
- Multi-joint Coordination: Grouping muscles/joints into "synergies" to simplify control.
- Temporal Abstraction: High-level layers plan over seconds/minutes, while low-level layers act in milliseconds.
Methods and Architectures
A state-of-the-art example of these principles is the Neural Probabilistic Motor Primitives (NPMP) architecture. Here, a low-level controller is first trained on human motion-capture (MoCap) data to learn "how to move," and a high-level controller is then added to learn "what to do."
A critical technical insight is the bottleneck benefit. In AI, forcing a "low-bandwidth" connection between a high-level and low-level controller (an inter-layer bottleneck) prevents the high-level "brain" from micromanaging. This forces the low-level "body" to become more robust and autonomous, mirroring biological findings where decorticate animals can still perform complex locomotor tasks.
Key Results and Neuroanatomical Mapping
Hierarchical models can solve complex navigation and interaction tasks in simulated environments that "flat" RL agents simply cannot learn. The authors successfully map their six AI principles onto the mammalian brain:
- Spine/Brainstem: Implements partial autonomy and multi-joint coordination.
- Basal Ganglia: Facilitates the selection of "amortized" motor programs (habits).
- Motor Cortex: Provides high-level refinement and "dexterity" for novel tasks.
Limitations and Future Work
Despite progress, several challenges remain. We still do not fully understand the "language" the brain uses to communicate between layers—whether it sends goals, predictions of sensory consequences, or error signals. Additionally, while current hierarchical AI models excel at locomotion, they still struggle with complex object manipulation and the "one-shot" imitation abilities seen in humans and songbirds.
Summary updated