On-line policy optimisation of Bayesian spoken dialogue systems via human interaction

Milica Gasic, Catherine Breslin, Matthew Henderson, Dongho Kim, Martin Szummer, Blaise Thomson, Pirros Tsiakoulis, Steve J. YoungICASSP 2013 · OpenReview

Abstract

A partially observable Markov decision process has been proposed as a dialogue model that enables robustness to speech recognition errors and automatic policy optimisation using reinforcement learning (RL). However, conventional RL algorithms require a very large number of dialogues, necessitating a user simulator. Recently, Gaussian processes have been shown to substantially speed up the optimisation, making it possible to learn directly from interaction with human users. However, early studies have been limited to very low dimensional spaces and the learning has exhibited convergence problems. Here we investigate learning from human interaction using the Bayesian Update of Dialogue State system. This dynamic Bayesian network based system has an optimisation space covering more than one hundred features, allowing a wide range of behaviours to be learned. Using an improved policy model and a more robust reward function, we show that stable learning can be achieved that significantly outperforms a simulator trained policy.
BibTeX
@inproceedings{gasic2013line,
  author = {Milica Gasic and Catherine Breslin and Matthew Henderson and Dongho Kim and Martin Szummer and Blaise Thomson and Pirros Tsiakoulis and Steve J. Young},
  title = {{On-line policy optimisation of Bayesian spoken dialogue systems via human interaction}},
  booktitle = {IEEE International Conference on Acoustics, Speech and Signal Processing},
  year = {2013},
  url = {https://openreview.net/forum?id=YGMHVpBeNmW},
}

Loading related content...

Figure