Autonomous shaping: knowledge transfer in reinforcement learning

George Konidaris, Andrew G. Barto

Abstract

We introduce the use of learned shaping rewards in reinforcement learning tasks, where an agent uses prior experience on a sequence of tasks to learn a portable predictor that estimates intermediate rewards, resulting in accelerated learning in later tasks that are related but distinct. Such agents can be trained on a sequence of relatively easy tasks in order to develop a more informative measure of reward that can be transferred to improve performance on more difficult tasks without requiring a hand coded shaping function. We use a rod positioning task to show that this significantly improves performance even after a very brief training period.
BibTeX
@inproceedings{konidaris2006autonomous,
  author = {George Konidaris and Andrew G. Barto},
  title = {{Autonomous shaping: knowledge transfer in reinforcement learning}},
  booktitle = {International Conference on Machine Learning},
  year = {2006},
  doi = {10.1145/1143844.1143906},
}

Loading related content...

Figure