Learning-based Hierarchical Control: Emulating the Central Nervous System for Bio-Inspired Legged Robot Locomotion

Ge Sun, Milad Shafiee, Peizhuo Li, Guillaume Bellegarda, Auke Jan Ijspeert, Guillaume Sartoretti

Abstract

Biological organisms navigate complex environments with remarkable ease, despite inherent neural conduction delays. This paper explores a hierarchical control framework for legged robots that emulates the mammalian central nervous system (CNS). The architecture separates basic rhythmic gait generation, handled by a "spinal policy," from high-level environmental adaptation, managed by a "descending modulation policy." Using Reinforcement Learning, the system was trained to traverse challenging terrains, including high stairs and wide gaps. Results indicate that this division of labor not only enables agile locomotion but also provides a "rhythm buffer" that enhances robustness against sensorimotor latencies, a critical factor for both biological and robotic systems.

Introduction

In the natural world, animals possess the extraordinary ability to move across treacherous terrain with fluid agility. This capability persists despite significant "conduction delays"—the time required for nerve impulses to travel between the brain, spinal cord, and muscles. In contrast, traditional robotic controllers often struggle with latency, typically relying on monolithic neural networks that attempt to manage balance, rhythm, and obstacle avoidance simultaneously. Such "end-to-end" approaches often lack the robustness seen in biological systems.

To address these limitations, researchers have looked toward the biological "division of labor" within the CNS. In mammals, the spinal cord contains Central Pattern Generators (CPGs) that produce basic rhythmic movements, while higher brain centers provide descending signals to modulate these rhythms based on sensory input.

Conceptual mapping between the biological central nervous system and the proposed hierarchical robot control framework.

Methods

The proposed framework utilizes a two-layer hierarchical architecture designed to mimic this biological structure.

  1. The Spinal Policy: Acting as the artificial spinal cord, this layer combines a Multi-Layer Perceptron (MLP) with nonlinear phase oscillators (CPGs). It focuses on the fundamental mechanics of walking, adjusting CPG parameters such as frequency and amplitude based on proprioceptive feedback from the robot's Inertial Measurement Unit (IMU) and joint states.
  2. The Descending Modulation Policy: Emulating the motor cortex, this layer receives proprioceptive data alongside a visual Terrain Height Map. Rather than generating a new rhythm, it "fine-tunes" the spinal output by adding spatial offsets ($x$ and $z$ coordinates) to the foot positions, allowing for precise obstacle negotiation.
Detailed block diagram of the hierarchical control system, illustrating the interaction between the CPG-based spinal cord, descending modulation, and the robot's internal kinematic model.

The system was trained using Proximal Policy Optimization (PPO) within the Isaac Gym simulation environment. Training occurred in two stages: first, the spinal policy was optimized for flat-ground walking; second, with the spinal policy frozen, the descending policy was trained to navigate complex obstacles, including stairs up to 18cm and hurdles up to 30cm.

Results and Discussion

The hierarchical controller demonstrated exceptional versatility, successfully traversing all evaluation terrains. The robot crossed 25cm gaps and climbed obstacles nearly as tall as its own body height (30cm) without falling across multiple trials.

A key finding was the system's inherent robustness to sensorimotor delays. The spinal policy acted as a "rhythm buffer," maintaining stability even with IMU delays of up to 330ms—equivalent to a full gait cycle. This suggests that rhythmic, CPG-based control allows a robot to continue moving safely even when sensory data is outdated. However, the descending modulation policy proved more sensitive to latency; precise obstacle negotiation required low-latency visual feedback, with failures occurring at delays as low as 30ms in the descending pathway.

Furthermore, the study observed an interesting "sync" effect: feedback delayed by exactly one gait cycle performed better than feedback delayed by a half-cycle, as the old data aligned with the current rhythmic state of the limbs.

Conclusion

This research validates the biological hypothesis that a multi-layered, hierarchical control structure is essential for robust and agile locomotion in the presence of "slow" neural or electronic components. Future work will focus on deploying this framework on physical hardware and exploring concurrent training of both layers to further optimize the synergy between rhythmic stability and precise modulation.

Summary updated

BibTeX
@inproceedings{sun2024learning,
  author = {Ge Sun and Milad Shafiee and Peizhuo Li and Guillaume Bellegarda and Auke Jan Ijspeert and Guillaume Sartoretti},
  title = {{Learning-based Hierarchical Control: Emulating the Central Nervous System for Bio-Inspired Legged Robot Locomotion}},
  booktitle = {IEEE/RSJ International Conference on Intelligent Robots and Systems},
  year = {2024},
  url = {https://openreview.net/forum?id=3LIiqegauY},
}
Figure