Reinforcement Learning for Sparse-Reward Object-Interaction Tasks in a First-person Simulated 3D Environment
Lists
Only you see this
Abstract
Learning how to execute complex tasks involving multiple objects in a 3D world is challenging when there is no ground-truth information about the objects or any demonstration to learn from. When an agent only receives a signal from task-completion, this makes it challenging to learn the object-representations which support learning the correct object-interactions needed to complete the task. In this work, we formulate learning an attentive object dynamics model as a classification problem, using random object-images to define incorrect labels for our object-dynamics model. We show empirically that this enables object-representation learning that captures an object's category (is it a toaster?), its properties (is it on?), and object-relations (is something inside of it?). With this, our core learner (a relational RL agent) receives the dense training signal it needs to rapidly learn object-interaction tasks. We demonstrate results in the 3D AI2Thor simulated kitchen environment with a range of challenging food preparation tasks. We compare our method's performance to several related approaches and against the performance of an oracle: an agent that is supplied with ground-truth information about objects in the scene. We find that our agent achieves performance closest to the oracle in terms of both learning speed and maximum success rate.
BibTeX
@inproceedings{carvalho2021reinforcement,
author = {Wilka Carvalho and Anthony Liang and Kimin Lee and Sungryull Sohn and Honglak Lee and Richard L. Lewis and Satinder Singh},
title = {{Reinforcement Learning for Sparse-Reward Object-Interaction Tasks in a First-person Simulated 3D Environment}},
booktitle = {International Joint Conference on Artificial Intelligence},
year = {2021},
doi = {10.24963/ijcai.2021/306},
}