Online BayesSim for Combined Simulator Parameter Inference and Policy Improvement

Rafael Possas, Lucas Barcelos, Rafael Oliveira, Dieter Fox, Fabio Ramos

Abstract

Recent advancements in Bayesian likelihood-free inference enables a probabilistic treatment for the problem of estimating simulation parameters and their uncertainty given sequences of observations. Domain randomization can be performed much more effectively when a posterior distribution provides the correct uncertainty over parameters in a simulated environment. In this paper, we study the integration of simulation parameter inference with both model-free reinforcement learning and model-based control in a novel sequential algorithm that alternates between learning a better estimation of parameters and improving the controller. This approach exploits the interdependence between the two problems to generate computational efficiencies and improved reliability when a black-box simulator is available. Experimental results suggest that both control strategies have better performance when compared to traditional domain randomization methods.
BibTeX
@inproceedings{possas2020online,
  author = {Rafael Possas and Lucas Barcelos and Rafael Oliveira and Dieter Fox and Fabio Ramos},
  title = {{Online BayesSim for Combined Simulator Parameter Inference and Policy Improvement}},
  booktitle = {IEEE/RSJ International Conference on Intelligent Robots and Systems},
  year = {2020},
  doi = {10.1109/iros45743.2020.9341401},
}

Loading related content...

Figure