Improving the Morphology and Control Policy of Self-reconfiguring Modular Robots in Dynamic Environment (Student Abstract)

Dong Xue, Qiang Lu, Jian LiAAAI 2021 · DOI

Introduction

Self-reconfiguring modular robots (SMRs) are designed to be the "chameleons" of robotics, capable of altering their physical structure to navigate diverse terrains such as stairs, pipes, or gaps. However, most existing SMRs are optimized for static, pre-defined environments. When faced with dynamic environments—sequences of unknown and varying terrains—these robots often struggle. Traditional evolutionary approaches like Genetic Algorithms (GA) are computationally expensive and slow, as they require physical testing of every candidate morphology and control policy. This paper introduces the Two-Stage Reconfiguration Algorithm (TSRA) to address these efficiency gaps.

Methodology: The Two-Stage Reconfiguration Algorithm (TSRA)

The core of the proposed approach is a co-evolutionary process that adapts both the robot's "body" (morphology $\beta$) and "brain" (control policy $\zeta$). TSRA operates in two distinct phases to minimize costly real-world interactions:

  1. Experience Reconfiguration (ER): This phase serves as a "mental rehearsal." A Genetic Algorithm generates candidate configurations, which are then evaluated by an Estimation Network (a neural surrogate model). This allows the system to rapidly screen thousands of possibilities without physical execution.
  2. Actual Reconfiguration (AR): The highest-performing candidates from the ER phase are deployed in the physical (or simulated) environment. The performance data gathered during this "real-world" phase is then used to update the Estimation Network, refining its predictive accuracy for future scenes.
The Two-Stage Reconfiguration Algorithm (TSRA) workflow, illustrating the interaction between the Estimation Network and physical evaluation.

Experimental Setup

The algorithm was evaluated using the Unity3D engine and ML-Agents toolkit. Two benchmark "Dynamic Environments" were created, each consisting of a sequence of four distinct, unknown terrains. TSRA was compared against two baselines:

  • RE-GA: A standard Genetic Algorithm without the estimation stage.
  • CEM-RL: A Cross-Entropy Method Reinforcement Learning approach.

Key Results and Discussion

The experiments demonstrated that TSRA significantly improves the efficiency of the reconfiguration process. While both TSRA and RE-GA eventually successfully navigated the environments, TSRA reached the goal much faster due to its ability to "imagine" successful configurations.

Performance comparison in Dynamic Environment 1, showing TSRA's superior convergence speed compared to RE-GA and CEM-RL.

In simulation, TSRA enabled the SMR to traverse complex dynamic environments in approximately 170 steps. Notably, the reinforcement learning baseline (CEM-RL) failed to effectively discover viable morphologies, suggesting that evolutionary strategies enhanced by surrogate models are more robust for structural adaptation tasks.

Limitations and Future Work

While promising, the current study is limited to simulation. Real-world deployment will introduce hardware constraints such as joint torque limits and mechanical wear. Future research will focus on refining the encoding of morphology and actions to support more complex 3D behaviors and scaling the system to a larger number of robotic modules.

Summary updated

BibTeX
@inproceedings{xue2021improving,
  author = {Dong Xue and Qiang Lu and Jian Li},
  title = {{Improving the Morphology and Control Policy of Self-reconfiguring Modular Robots in Dynamic Environment (Student Abstract)}},
  booktitle = {AAAI Conference on Artificial Intelligence},
  year = {2021},
  doi = {10.1609/aaai.v35i18.17966},
}
Figure