Data-efficient Learning of Morphology and Controller for a Microrobot

Thomas Liao, Grant Wang, Brian H. Yang, Rene Lee, Kristofer S. J. Pister, Sergey Levine, Roberto Calandra

TL;DR

This paper introduces HPC-BBO, a novel hierarchical Bayesian optimization approach that simultaneously learns the physical design (morphology) and control strategy for microrobots. By efficiently co-optimizing these aspects and leveraging batch manufacturing, HPC-BBO drastically reduces the time and resources needed to develop high-performing robots, achieving the same results in a fifth of the time compared to standard methods.

What Problem Does This Paper Address?

Designing robots, especially at the micro-scale, is a slow and expensive process. Traditionally, robot design and controller development are treated as separate stages. This often leads to either general-purpose designs that are suboptimal for specific tasks or an iterative cycle where a new hardware design is followed by controller tuning, requiring significant expert knowledge and time for each iteration. For microrobots, this is exacerbated by the long manufacturing lead times (weeks for a single wafer of robots) and the limited lifespan of the robots, necessitating efficient design and control optimization within a few attempts. The paper tackles the challenge of creating a data-efficient optimization framework that can jointly learn both the robot's physical form and its control policy, minimizing the need for extensive physical prototypes and expert intervention.

Methods

The core contribution is the Hierarchical Process Constrained Batch Bayesian Optimization (HPC-BBO) algorithm. This method addresses the problem by:

  • Hierarchical Optimization: It separates the optimization into two levels: morphology optimization (the "process-constrained" part, as it's slow and expensive) and controller optimization (which is relatively cheap).
  • Batch Manufacturing: HPC-BBO leverages the ability to manufacture multiple robots (morphologies) on a single silicon wafer simultaneously. It selects a batch of promising morphology designs at once.
  • Bayesian Optimization (BO): Both levels of optimization utilize Bayesian Optimization.
  • Morphology Optimization: Batch Bayesian Optimization (BBO) is used to select multiple morphology candidates in parallel. This includes a "hallucination" step where the GP model predicts rewards for un-evaluated morphologies to guide the selection of the next batch.
  • Controller Optimization: Contextual Bayesian Optimization (cBO) is employed to optimize the controller for each selected morphology. The morphology is treated as a "context," allowing the controller optimization to generalize better across similar designs.
  • Data Efficiency: By using the learned GP models and the contextual information, HPC-BBO efficiently explores the morphology space and reuses knowledge from previous controller optimizations, significantly reducing the number of expensive morphology evaluations needed.

Datasets

The paper uses a simulated 6-legged microrobot in the V-REP simulator for evaluation. The simulator is used to mimic the real-world scenario where physical evaluation is costly and time-consuming. No new datasets are introduced; rather, the simulation environment serves as a proxy for real-world experiments.

Key Insights and Ideas

  • Morphology and Control are Intertwined: The results strongly suggest that optimal robot performance is achieved by jointly optimizing morphology and control, rather than optimizing them independently. Recombining a highly performing controller with a slightly different morphology can lead to drastic performance drops (up to 90%).
  • Batching is Crucial for Speed: The ability to evaluate multiple morphologies in parallel (batching) is the primary driver of the significant speedup in the optimization process. This directly translates to fewer fabrication cycles and thus less wall-clock time.
  • Contextual Optimization Boosts Early Performance: Treating morphology as a context for controller optimization (cBO) allows the algorithm to learn more generalizable controllers and adapt faster to new, unseen morphologies, especially in the early stages of optimization.
  • Hierarchical Structure is Key: The hierarchical approach, combined with batching, effectively manages the differing costs of evaluating morphologies versus controllers, leading to a more efficient overall optimization strategy.

Key Results

  • 360% Improvement: Contextual HPC-BBO achieved a 360% improvement over standard Bayesian Optimization in terms of performance achieved per fabrication cycle.
  • Time Reduction: The proposed approach reduced the hypothetical manufacturing time from 21 months (for standard BO) to 4 months to reach the same performance level. This is a reduction to one-fifth of the time.
  • Batching Efficiency: HPC-BBO significantly outperforms baselines (Random Search, CMA-ES, Standard BO) in terms of fabrication cycles and morphologies evaluated.
  • Controller Specificity: Table I demonstrates that controllers are highly specific to their morphologies, with performance dropping significantly when controllers are swapped between different designs.

Limitations and Future Work

  • Simulation Dependency: While the method is designed to be data-efficient for real-world application, the primary validation was performed in simulation. The direct translation of simulation results to real-world performance is not explicitly quantified.
  • GP Model Limitations: The performance of Bayesian Optimization relies on the accuracy of the Gaussian Process model. For highly complex or multi-modal objective functions, the GP might struggle to accurately model the landscape.
  • Future Directions:
    • "Opening the Black-Box": Replacing Bayesian Optimization with model-based reinforcement learning for more complex controllers.
  • Real-World Application: Applying the proposed approach to actual microrobot hardware.
  • More Complex Controllers: Extending the framework to handle more intricate control strategies beyond CPGs.

Summary updated

BibTeX
@inproceedings{liao2019data,
  author = {Thomas Liao and Grant Wang and Brian H. Yang and Rene Lee and Kristofer S. J. Pister and Sergey Levine and Roberto Calandra},
  title = {{Data-efficient Learning of Morphology and Controller for a Microrobot}},
  booktitle = {IEEE International Conference on Robotics and Automation},
  year = {2019},
  doi = {10.1109/icra.2019.8793802},
}

Loading related content...

Figure