Variational Auto-Regularized Alignment for Sim-to-Real Control

Martin Hwasser, Danica Kragic, Rika Antonova

Abstract

General-purpose simulators can be a valuable data source for flexible learning and control approaches. However, training models or control policies in simulation and then directly applying to hardware can yield brittle control. Instead, we propose a novel way to use simulators as regularizers. Our approach regularizes a decoder of a variational autoencoder to a black-box simulation, with the latent space bound to a subset of simulator parameters. This enables successful encoder training from a small number of real-world trajectories (10 in our experiments), yielding a latent space with simulation parameter distribution that matches the real-world setting. We use a learnable mixture for the latent prior/posterior, which implies a highly flexible class of densities for the posterior fit. Our approach is scalable and does not require restrictive distributional assumptions. We demonstrate ability to recover matching parameter distributions on a range of benchmarks, challenging custom simulation environments and several real-world scenarios. Our experiments using ABB YuMi robot hardware show ability to help reinforcement learning approaches overcome cases of severe sim-to-real mismatch.
BibTeX
@inproceedings{hwasser2020variational,
  author = {Martin Hwasser and Danica Kragic and Rika Antonova},
  title = {{Variational Auto-Regularized Alignment for Sim-to-Real Control}},
  booktitle = {IEEE International Conference on Robotics and Automation},
  year = {2020},
  doi = {10.1109/icra40945.2020.9197130},
}

Loading related content...

Figure