Robots that can adapt like natural animals.

Antoine Cully, Jeff Clune, Jean-Baptiste MouretarXiv 2014 · DOI · OpenReview

TL;DR

Robots can now adapt to damage in minutes, much like animals, thanks to a novel "Intelligent Trial and Error" algorithm. This approach uses a pre-computed "behavior-performance map" to guide rapid, adaptive learning, enabling robots to recover from injuries without pre-programmed contingency plans.

What Problem Does This Paper Address?

As robots move beyond controlled factory settings into complex, real-world environments (think search and rescue, exploration), their fragility becomes a major hurdle. Unlike animals, which can intuitively adapt to injuries by finding new ways to move or function, current robots are brittle. They rely on expensive self-diagnosis systems and pre-programmed responses for every conceivable failure. This approach is impractical for complex robots, as it's impossible to anticipate every potential damage scenario. The paper aims to bridge this gap by enabling robots to learn compensatory behaviors quickly and creatively after damage occurs, mimicking animal resilience.

Methods

The core of the proposed solution is the "Intelligent Trial and Error" (IT&E) algorithm, which has two main phases:

  1. Behavior-Performance Map Creation (MAP-Elites): Before deployment, a robot uses a novel algorithm called MAP-Elites to explore its potential behaviors in simulation. It creates a "behavior-performance map" that essentially encodes the robot's "intuition" about which behaviors are likely to perform well in different scenarios. This map is a low-dimensional representation of a high-dimensional behavior space, making it more manageable. This step is computationally intensive but is performed only once per robot design.
  2. Adaptation Step (Map-based Bayesian Optimization - M-BOA): When the robot encounters damage and its performance drops, it uses the pre-computed behavior-performance map to guide its learning. M-BOA, a Bayesian optimization algorithm, leverages the map's predictions as a prior. It intelligently selects behaviors to test on the physical robot, balancing exploration (trying uncertain behaviors) and exploitation (trying behaviors predicted to be good). After each test, the robot updates its map with the actual performance, rapidly discovering a compensatory behavior.

Datasets

The paper does not explicitly mention using pre-existing datasets for training. Instead, it relies on:

  • Simulated Robot Data: The behavior-performance maps are created using physics simulators (Open Dynamics Engine for the hexapod, and a custom simulator for the robotic arm). Millions of simulated behaviors are evaluated to build these maps.
  • Physical Robot Experiments: The adaptation phase is tested on physical hexapod robots and robotic arms. These experiments serve as the evaluation benchmarks.

Key Insights and Ideas

  • Animals as Inspiration: The paper draws a strong parallel between how animals intuitively adapt to injury and the proposed IT&E algorithm. This suggests that nature's solutions can inform robust robotic design.
  • "Intuition" via Behavior-Performance Maps: Pre-computing a map of high-performing behaviors allows robots to have a form of "intuition" about their capabilities, guiding them more effectively than random exploration.
  • Low-Dimensionality is Key: Reducing the vast space of possible robot behaviors to a lower-dimensional "behavior space" is crucial for efficient learning and adaptation.
  • Bayesian Optimization with Priors: Using a pre-computed map as a prior for Bayesian optimization significantly speeds up the adaptation process compared to starting from scratch.
  • Damage Recovery Without Diagnosis: The system doesn't need to understand why it's damaged; it just needs to find a way to perform despite the damage.

Key Results

  • Rapid Adaptation: Robots can adapt to damage in less than two minutes, often in under 30 seconds, with fewer than 10 trials.
  • Successful Adaptations: The algorithm successfully adapted a hexapod robot injured in five different ways (including broken and missing legs) and a robotic arm with joints broken in 14 different ways.
  • Performance Improvement: Compensatory gaits for the hexapod robot were 3-7 times faster than a standard reference gait after damage.
  • Robustness: The algorithm showed robustness to environmental changes (sloped terrains) and performed well across various behavioral descriptor choices.
  • Superiority over Baselines: IT&E significantly outperformed traditional Bayesian optimization and policy gradient methods, especially in scenarios with limited trials or high-dimensional search spaces.

Limitations and Future Work

  • Behavioral Descriptor Choice: While the algorithm is robust to different descriptors, the performance can be further improved with prior knowledge about which dimensions are most informative. The paper notes that if the map doesn't contain a suitable behavior for a severe damage scenario, adaptation might be limited.
  • Computational Cost of Map Creation: The initial creation of the behavior-performance map is computationally expensive, requiring millions of simulations.
  • Future Work: The authors suggest exploring more sophisticated behavioral descriptors, automating the damage detection threshold, and investigating how sleep might inspire creativity in robots (a parallel drawn from animal cognition). They also highlight the potential for this approach to enable more robust and effective autonomous robots in general.

Summary updated

Abstract

As robots leave the controlled environments of factories to autonomously function in more complex, natural environments, they will have to respond to the inevitable fact that they will become damaged. However, while animals can quickly adapt to a wide variety of injuries, current robots cannot "think outside the box" to find a compensatory behavior when damaged: they are limited to their pre-specified self-sensing abilities, can diagnose only anticipated failure modes, and require a pre-programmed contingency plan for every type of potential damage, an impracticality for complex robots. Here we introduce an intelligent trial and error algorithm that allows robots to adapt to damage in less than two minutes, without requiring self-diagnosis or pre-specified contingency plans. Before deployment, a robot exploits a novel algorithm to create a detailed map of the space of high-performing behaviors: This map represents the robot's intuitions about what behaviors it can perform and their value. If the robot is damaged, it uses these intuitions to guide a trial-and-error learning algorithm that conducts intelligent experiments to rapidly discover a compensatory behavior that works in spite of the damage. Experiments reveal successful adaptations for a legged robot injured in five different ways, including damaged, broken, and missing legs, and for a robotic arm with joints broken in 14 different ways. This new technique will enable more robust, effective, autonomous robots, and suggests principles that animals may use to adapt to injury.
BibTeX
@misc{cully2014robots,
  author = {Antoine Cully and Jeff Clune and Jean-Baptiste Mouret},
  title = {{Robots that can adapt like natural animals.}},
  year = {2014},
  eprint = {1407.3501},
  archivePrefix = {arXiv},
  url = {https://arxiv.org/abs/1407.3501},
}

Loading related content...

Figure