On Self-Adaptive Mutation Restarts for Evolutionary Robotics with Real Rotorcraft

Gerard David HowardarXiv 2017 · arXiv

Introduction

Evolutionary Robotics (ER) applies Darwinian principles to "grow" robot controllers, typically within simulated environments. While simulation offers a safe and fast testing ground, it often fails to account for the complex, "messy" physics of the real world—a phenomenon known as the "reality gap." Consequently, controllers optimized in simulation frequently underperform or fail when deployed on physical hardware.

To bridge this gap, researchers are increasingly evolving controllers directly on robot hardware. However, real-world evolution is constrained by time and the risk of hardware failure. While Self-Adaptation (SA) allows algorithms to automatically tune mutation and crossover rates, it often leads to fitness stagnation. In these scenarios, the population becomes trapped in a local optimum with suboptimal mutation rates, lacking the exploratory "spark" needed to escape. This study investigates the use of "restart" mechanisms to revitalize the evolutionary process when progress stalls.

Experimental Methodology

The researchers employed a specialized testing platform designed for 24/7 automated evolution. A real hexacopter was operated within a safety cage, tethered to provide continuous power and prevent crashes.

Experimental setup showing the tethered hexacopter and the fan providing wind disturbances.

The Evolutionary Algorithm

The study utilized Differential Evolution (DE), a robust population-based optimizer, to tune an 18-parameter PID (Proportional-Integral-Derivative) controller. Four distinct strategies were compared:

  1. STATIC: Fixed mutation and crossover rates (baseline).
  2. ADAPT: Self-adaptive rates that adjust based on success without resetting.
  3. INDIV (Individual Restarts): Mutation parameters for a specific individual are randomized if its fitness fails to improve for five generations.
  4. GLOBAL (Population Restarts): All individuals' mutation parameters are randomized if the population's best fitness stagnates for five generations.

Task and Environment

The "dataset" for this hardware-in-the-loop study consisted of a 40-second hovering sequence across five waypoints. To ensure robustness, an oscillating fan introduced wind gusts of approximately 5 m/s, forcing the evolved controllers to handle significant external disturbances.

Results and Analysis

The experimental results highlight a clear advantage for self-adaptive strategies over static baselines, with restart mechanisms providing critical reliability.

Convergence Speed and Reliability

All self-adaptive methods significantly outperformed the STATIC baseline in terms of convergence speed. As shown in the performance data, the GLOBAL strategy was the fastest on average (24.4 generations), followed closely by INDIV (27.6 generations). In contrast, the STATIC baseline required an average of 70.5 generations.

Comparison of the number of generations required to solve the task across different evolutionary strategies.

Crucially, restarts acted as an "insurance policy" against stagnation. Without restarts (the ADAPT strategy), some evolutionary runs became trapped in parameter ruts, taking nearly 200 generations to converge. The introduction of restarts effectively eliminated these outliers, making the process more predictable and efficient.

The Superiority of Individual Restarts

While GLOBAL restarts facilitated rapid convergence, the INDIV strategy achieved the highest peak fitness ($f = 127,185.2$), statistically outperforming both GLOBAL and standard ADAPT methods. The researchers suggest that individual-level restarts allow for a more context-sensitive search; by only resetting individuals that are stuck, the algorithm preserves the progress of others exploring different regions of the solution space.

Analysis of the differential weight ($F$) revealed that restarts maintain higher mean values for this parameter. This keeps the population "jumping" across the search space, preventing the premature convergence that often plagues real-world evolutionary robotics.

Conclusion and Future Work

This research demonstrates that individual-level mutation restarts are a powerful tool for evolving high-performing robot controllers directly on hardware. By preventing fitness stagnation, these mechanisms ensure that evolution remains productive even in complex, disturbed environments.

Future research will explore whether these benefits extend to more complex, multimodal tasks—such as acrobatic maneuvers—where the fitness landscape is likely more rugged. Additionally, the authors aim to verify if the advantages of individual-level restarts hold true for other robotic platforms, including legged or underwater vehicles.

Summary updated

Abstract

Self-adaptive parameters are increasingly used in the field of Evolutionary Robotics, as they allow key evolutionary rates to vary autonomously in a context-sensitive manner throughout the optimisation process. A significant limitation to self-adaptive mutation is that rates can be set unfavourably, which hinders convergence. Rate restarts are typically employed to remedy this, but thus far have only been applied in Evolutionary Robotics for mutation-only algorithms. This paper focuses on the level at which evolutionary rate restarts are applied in population-based algorithms with more than 1 evolutionary operator. After testing on a real hexacopter hovering task, we conclude that individual-level restarting results in higher fitness solutions without fitness stagnation, and population restarts provide a more stable rate evolution. Without restarts, experiments can become stuck in suboptimal controller/rate combinations which can be difficult to escape from.
BibTeX
@misc{howard2017self,
  author = {Gerard David Howard},
  title = {{On Self-Adaptive Mutation Restarts for Evolutionary Robotics with Real Rotorcraft}},
  year = {2017},
  eprint = {1703.10754},
  archivePrefix = {arXiv},
  url = {https://arxiv.org/abs/1703.10754},
}
Figure