Introduction
Self-reconfiguring modular robots (SMRs) are designed to be the "chameleons" of robotics, capable of altering their physical structure to navigate diverse terrains such as stairs, pipes, or gaps. However, most existing SMRs are optimized for static, pre-defined environments. When faced with dynamic environments—sequences of unknown and varying terrains—these robots often struggle. Traditional evolutionary approaches like Genetic Algorithms (GA) are computationally expensive and slow, as they require physical testing of every candidate morphology and control policy. This paper introduces the Two-Stage Reconfiguration Algorithm (TSRA) to address these efficiency gaps.
Methodology: The Two-Stage Reconfiguration Algorithm (TSRA)
The core of the proposed approach is a co-evolutionary process that adapts both the robot's "body" (morphology $\beta$) and "brain" (control policy $\zeta$). TSRA operates in two distinct phases to minimize costly real-world interactions:
- Experience Reconfiguration (ER): This phase serves as a "mental rehearsal." A Genetic Algorithm generates candidate configurations, which are then evaluated by an Estimation Network (a neural surrogate model). This allows the system to rapidly screen thousands of possibilities without physical execution.
- Actual Reconfiguration (AR): The highest-performing candidates from the ER phase are deployed in the physical (or simulated) environment. The performance data gathered during this "real-world" phase is then used to update the Estimation Network, refining its predictive accuracy for future scenes.
Experimental Setup
The algorithm was evaluated using the Unity3D engine and ML-Agents toolkit. Two benchmark "Dynamic Environments" were created, each consisting of a sequence of four distinct, unknown terrains. TSRA was compared against two baselines:
- RE-GA: A standard Genetic Algorithm without the estimation stage.
- CEM-RL: A Cross-Entropy Method Reinforcement Learning approach.
Key Results and Discussion
The experiments demonstrated that TSRA significantly improves the efficiency of the reconfiguration process. While both TSRA and RE-GA eventually successfully navigated the environments, TSRA reached the goal much faster due to its ability to "imagine" successful configurations.
In simulation, TSRA enabled the SMR to traverse complex dynamic environments in approximately 170 steps. Notably, the reinforcement learning baseline (CEM-RL) failed to effectively discover viable morphologies, suggesting that evolutionary strategies enhanced by surrogate models are more robust for structural adaptation tasks.
Limitations and Future Work
While promising, the current study is limited to simulation. Real-world deployment will introduce hardware constraints such as joint torque limits and mechanical wear. Future research will focus on refining the encoding of morphology and actions to support more complex 3D behaviors and scaling the system to a larger number of robotic modules.
Summary updated