Improved Policy Extraction via Online Q-Value Distillation
Lists
Only you see this
Abstract
Deep neural networks are capable of solving complex control tasks in challenging environments, but their learned policies are hard to interpret. Not being able to explain or verify them limits their practical applicability. By contrast, decision trees lend themselves well to explanation and verification, but are not easy to train, especially in an online fashion. In this work we introduce Q-BSP trees and propose an Ordered Sequential Monte Carlo training algorithm that efficiently distills the Q-function from fully trained deep Q-networks into a tree structure. Q-BSP forests are used to generate the partitioning rules that transparently reconstruct an accurate value function. We explain our approach and provide results that convincingly beat earlier online policy distillation methods with respect to their own performance benchmarks.
BibTeX
@inproceedings{jhunjhunwala2020improved,
author = {Aman Jhunjhunwala and Jaeyoung Lee and Sean Sedwards and Vahdat Abdelzad and Krzysztof Czarnecki},
title = {{Improved Policy Extraction via Online Q-Value Distillation}},
booktitle = {IJCNN},
year = {2020},
url = {https://openreview.net/forum?id=1QEdg54V9c0d},
}Loading related content...