Introduction
Evolutionary Robotics (ER) applies Darwinian principles to "grow" robot controllers, typically within simulated environments. While simulation offers a safe and fast testing ground, it often fails to account for the complex, "messy" physics of the real world—a phenomenon known as the "reality gap." Consequently, controllers optimized in simulation frequently underperform or fail when deployed on physical hardware.
To bridge this gap, researchers are increasingly evolving controllers directly on robot hardware. However, real-world evolution is constrained by time and the risk of hardware failure. While Self-Adaptation (SA) allows algorithms to automatically tune mutation and crossover rates, it often leads to fitness stagnation. In these scenarios, the population becomes trapped in a local optimum with suboptimal mutation rates, lacking the exploratory "spark" needed to escape. This study investigates the use of "restart" mechanisms to revitalize the evolutionary process when progress stalls.
Experimental Methodology
The researchers employed a specialized testing platform designed for 24/7 automated evolution. A real hexacopter was operated within a safety cage, tethered to provide continuous power and prevent crashes.
The Evolutionary Algorithm
The study utilized Differential Evolution (DE), a robust population-based optimizer, to tune an 18-parameter PID (Proportional-Integral-Derivative) controller. Four distinct strategies were compared:
- STATIC: Fixed mutation and crossover rates (baseline).
- ADAPT: Self-adaptive rates that adjust based on success without resetting.
- INDIV (Individual Restarts): Mutation parameters for a specific individual are randomized if its fitness fails to improve for five generations.
- GLOBAL (Population Restarts): All individuals' mutation parameters are randomized if the population's best fitness stagnates for five generations.
Task and Environment
The "dataset" for this hardware-in-the-loop study consisted of a 40-second hovering sequence across five waypoints. To ensure robustness, an oscillating fan introduced wind gusts of approximately 5 m/s, forcing the evolved controllers to handle significant external disturbances.
Results and Analysis
The experimental results highlight a clear advantage for self-adaptive strategies over static baselines, with restart mechanisms providing critical reliability.
Convergence Speed and Reliability
All self-adaptive methods significantly outperformed the STATIC baseline in terms of convergence speed. As shown in the performance data, the GLOBAL strategy was the fastest on average (24.4 generations), followed closely by INDIV (27.6 generations). In contrast, the STATIC baseline required an average of 70.5 generations.
Crucially, restarts acted as an "insurance policy" against stagnation. Without restarts (the ADAPT strategy), some evolutionary runs became trapped in parameter ruts, taking nearly 200 generations to converge. The introduction of restarts effectively eliminated these outliers, making the process more predictable and efficient.
The Superiority of Individual Restarts
While GLOBAL restarts facilitated rapid convergence, the INDIV strategy achieved the highest peak fitness ($f = 127,185.2$), statistically outperforming both GLOBAL and standard ADAPT methods. The researchers suggest that individual-level restarts allow for a more context-sensitive search; by only resetting individuals that are stuck, the algorithm preserves the progress of others exploring different regions of the solution space.
Analysis of the differential weight ($F$) revealed that restarts maintain higher mean values for this parameter. This keeps the population "jumping" across the search space, preventing the premature convergence that often plagues real-world evolutionary robotics.
Conclusion and Future Work
This research demonstrates that individual-level mutation restarts are a powerful tool for evolving high-performing robot controllers directly on hardware. By preventing fitness stagnation, these mechanisms ensure that evolution remains productive even in complex, disturbed environments.
Future research will explore whether these benefits extend to more complex, multimodal tasks—such as acrobatic maneuvers—where the fitness landscape is likely more rugged. Additionally, the authors aim to verify if the advantages of individual-level restarts hold true for other robotic platforms, including legged or underwater vehicles.
Summary updated