Snake Robot with Tactile Perception Navigates on Large-scale Challenging Terrain

Shuo Jiang, Adarsh Salagame, Alireza Ramezani, Lawson L. S. Wong

TL;DR

This paper introduces a hierarchical reinforcement learning (HRL) framework for snake robots that leverages whole-body tactile sensing to achieve adaptive locomotion on challenging, large-scale terrains. By integrating tactile feedback into a curriculum learning approach, the snake robot demonstrates significantly improved navigation efficiency and robustness in complex environments like caves.

What Problem Does This Paper Address?

Snake robots, with their unique serpentine form, are adept at navigating confined and complex spaces inaccessible to other robots. However, their locomotion on uneven or unpredictable terrain is a significant challenge. While tactile sensing offers rich information about the environment (e.g., surface roughness, slope), effectively integrating this data into control strategies for dynamic locomotion has been difficult. Existing methods often struggle with the computational cost of processing dense tactile data or fail to generalize to novel terrains. This paper aims to bridge this gap by developing a control system that uses tactile perception to enable snake robots to adapt their gaits in real-time for efficient navigation in complex, large-scale environments.

Methods

The proposed solution employs a hierarchical reinforcement learning (HRL) architecture.

  • Global Navigation: A high-level planner (A* algorithm) generates an efficient path, which is then segmented into waypoints.
  • Local Navigation: This is the core of the paper's contribution. It uses a two-phase reinforcement learning approach:
  • Curriculum Learning (Gait Library Generation): The robot first learns a library of basic gaits (e.g., sidewinding, lateral rolling) on a spectrum of procedurally generated terrains. This phase does not use tactile information, focusing solely on learning diverse locomotion patterns.
  • Tactile-Adaptive Local Navigation: In a second training phase, the robot learns to select and adapt gaits from the pre-trained library using local tactile information. This involves:
  • Adaptors: For each joint, an "Adaptor" module takes local tactile readings from adjacent sensors and selects a gait from the library using a one-hot encoding. This leverages the idea that local contact patterns are most relevant for immediate gait adjustments.
  • Centralized Training, Decentralized Execution (CTDE): This multi-agent RL paradigm is used to train the Adaptors, allowing for coordinated behavior while each Adaptor makes decisions based on local observations.
  • Distributed Training: To overcome the computational burden of simulating numerous tactile sensors, a distributed RL framework was developed. This involves a central server for training (critic and actor) and multiple worker workstations running simulator instances.

Datasets

The paper uses procedurally generated terrains for training and evaluation.

  • Curriculum Terrains: Perlin noise is used to generate a spectrum of terrains for the initial gait library learning phase.
  • Test Caves: The primary evaluation is conducted in randomly generated, large-scale cave environments (155m x 102m) with uneven surfaces, designed to challenge the robot's navigation capabilities. Multiple random cave layouts are used to test generalization.

Key Insights and Ideas

  • Tactile Information is Crucial for Real-time Adaptation: Simply adding tactile readings to the state space of a single RL agent (as in the "Tac" baseline) proved ineffective. The proposed hierarchical approach, which separates gait learning from tactile adaptation, is key.
  • Local Tactile Information is Sufficient for Gait Selection: The "Adaptor" modules, which only use tactile data from adjacent sensors, are sufficient for selecting appropriate gaits, demonstrating that global tactile awareness isn't always necessary for local adjustments.
  • Curriculum Learning Decouples Tasks: Separating the learning of basic gaits from the learning of tactile adaptation significantly simplifies the problem and improves learning efficiency.
  • Distributed Simulation is Necessary: The computational cost of simulating dense tactile sensors severely impacts simulator efficiency. The proposed distributed framework is essential for practical training.
  • One-Hot Gait Selection is Superior to Softmax: Using a one-hot selection for gaits from the library led to better performance than a weighted mixture, which tended to average out gaits and neglect tactile information.

Key Results

  • Superior Navigation Performance: The proposed tactile-adaptive HRL approach significantly outperformed baselines (including those using CPGs with or without domain randomization) in navigating complex cave environments.
  • Improved Efficiency: The paper reports improved "motion efficiency" in challenging terrains, though specific quantitative metrics like time to goal or distance traveled are presented in tables and figures. For instance, in Table II, the "Ours" approach shows significantly higher "return" (a measure of performance) in untrained test environments (T7, T8) compared to models trained only on curriculum data (M1-M6) or domain randomization (DR).
  • Generalization: The approach demonstrated the ability to adapt to entirely new, untrained terrains, a capability lacking in models that did not incorporate tactile feedback.
  • Simulation Efficiency: The distributed RL framework significantly mitigates the efficiency reduction caused by dense tactile sensing, as shown by the Real-Time Factor (RTF) comparisons in Table I.

Limitations and Future Work

  • Sim-to-Real Gap: The current validation is performed in simulation. The authors acknowledge that future work will focus on sim-to-real transfer and validation on real robot platforms.
  • Limited Tactile Integration: While the Adaptors use local tactile information, the critic receives global tactile patterns. Further exploration of how global tactile information can be more effectively utilized might be beneficial.
  • Computational Cost: Despite the distributed approach, complex simulations with many sensors remain computationally intensive.
  • Future Work: The authors explicitly mention sim-to-real transfer and real-world validation as key future directions.

Summary updated

Abstract

Along with the advancement of robot skin technology, there has been notable progress in the development of snake robots featuring body-surface tactile perception. In this study, we proposed a locomotion control framework for snake robots that integrates tactile perception to augment their adaptability to various terrains. Our approach embraces a hierarchical reinforcement learning (HRL) architecture, wherein the high-level orchestrates global navigation strategies while the low-level uses curriculum learning for local navigation maneuvers. Due to the significant computational demands of collision detection in whole-body tactile sensing, the efficiency of the simulator is severely compromised. Thus a distributed training pattern to mitigate the efficiency reduction was adopted. We evaluated the navigation performance of the snake robot in complex large-scale cave exploration with challenging terrains to exhibit improvements in motion efficiency, evidencing the efficacy of tactile perception in terrain-adaptive locomotion.
BibTeX
@inproceedings{jiang2024snake,
  author = {Shuo Jiang and Adarsh Salagame and Alireza Ramezani and Lawson L. S. Wong},
  title = {{Snake Robot with Tactile Perception Navigates on Large-scale Challenging Terrain}},
  booktitle = {IEEE International Conference on Robotics and Automation},
  year = {2024},
  url = {https://openreview.net/forum?id=ingSIRSSyG},
}
Figure