Efficient Automatic Tuning for Data-driven Model Predictive Control via Meta-Learning

Baoyu Li, William Edwards, Kris HauserarXiv 2024 · DOI · arXiv

Abstract

Model Predictive Control (MPC) is a cornerstone of modern robotics, yet its deployment is often hindered by the intensive manual effort required for system identification and hyper-parameter tuning. While frameworks like AutoMPC have automated this process using Bayesian Optimization (BO), they typically suffer from a "cold start" problem, requiring significant computational time for every new task. This paper introduces Portfolio, a meta-learning extension that "warmstarts" the tuning process using high-performing configurations from prior experiences. Our results show that Portfolio significantly accelerates convergence, improves model accuracy, and enhances control stability across a variety of robotic benchmarks.

Introduction

Setting up an effective MPC system involves choosing appropriate system identification (SysID) models, defining objective functions, and tuning numerous hyper-parameters. This process traditionally requires deep domain expertise and weeks of trial and error. The AutoMPC library was developed to automate these choices via BO. However, pure BO starts from scratch for every new robot or environment, leading to tuning sessions that can exceed 24 hours. Furthermore, this stochastic process can be unstable, occasionally converging to sub-optimal regions. Portfolio addresses these inefficiencies by allowing AutoMPC to leverage knowledge from past tuning tasks.

Methods: The Portfolio Approach

The Portfolio method, inspired by successful techniques in automated machine learning (AutoML), operates in two distinct phases:

  1. Offline Portfolio Building: AutoMPC is first run on a diverse set of "meta-datasets" (e.g., 20 different robotic simulations). The best-performing model configurations for each task are identified. A greedy algorithm then selects a diverse subset of these configurations to form the "Portfolio."
  2. Online Warmstarting: When a new, unseen task is encountered, AutoMPC evaluates the configurations in the Portfolio first. These high-quality starting points provide an immediate "warmstart" to the Bayesian Optimization process, guiding it toward promising areas of the search space from the very first iteration.
The two-phase Portfolio meta-learning approach: (a) offline portfolio construction from meta-datasets and (b) online application to the AutoMPC pipeline.

This approach is primarily applied to System Identification (SysID) tuning, the most computationally demanding component of the AutoMPC pipeline.

Experimental Evaluation

The study utilized 20 meta-training datasets derived from OpenAI Gym MuJoCo environments (e.g., Ant, Hopper, Walker2D) with various morphological and environmental modifications. Evaluation was conducted on 11 nonlinear control benchmarks, including out-of-distribution (OOD) tasks such as a physical underwater soft robot dataset.

Key Results

  • Accelerated Convergence: Portfolio consistently reaches lower error rates faster than pure BO. For instance, on the Walker2dSmall benchmark, Portfolio achieved target performance in half the iterations required by standard BO.
  • Improved Accuracy and Control: Portfolio achieved lower Root Mean Square Error (RMSE) across nearly all test datasets, with gains up to 6.9% in model accuracy. More importantly, this translated to better physical behavior; in the Cartpole benchmark, Portfolio-tuned models showed a 14.76% improvement in control performance.
  • Enhanced Stability: Unlike the stochastic nature of pure BO, Portfolio provides a deterministic set of high-quality initial points, making the tuning process more reliable and less prone to poor local minima.
Tuning curves comparing Pure Bayesian Optimization (BO) with the Portfolio-based approach on Walker2d and Hopper benchmarks.

Discussion and Conclusion

The primary strength of Portfolio lies in its ability to turn days of computation into hours by leveraging historical data. While the method shows significant gains when the new task shares characteristics with the meta-training set, its performance on highly novel domains—such as soft robotics—is currently comparable to pure BO.

Future work will focus on automatically determining optimal portfolio sizes for specific tasks and developing specialized portfolios for distinct robot classes. By making automated tuning more efficient and stable, Portfolio represents a significant step toward making advanced MPC accessible for a wider range of robotic applications.

Summary updated

BibTeX
@misc{li2024efficient,
  author = {Baoyu Li and William Edwards and Kris Hauser},
  title = {{Efficient Automatic Tuning for Data-driven Model Predictive Control via Meta-Learning}},
  year = {2024},
  eprint = {2404.00232},
  archivePrefix = {arXiv},
  url = {https://arxiv.org/abs/2404.00232},
}
Figure