Learning to Ask for Conversational Machine Learning

Shashank Srivastava, Igor Labutov, Tom M. MitchellEMNLP 2019 · OpenReview

Abstract

Natural language has recently been explored as a new medium of supervision for training machine learning models. Here, we explore learning classification tasks using language in a conversational setting – where the automated learner does not simply receive language input from a teacher, but can proactively engage the teacher by asking questions. We present a reinforcement learning framework, where the learner’s actions correspond to question types and the reward for asking a question is based on how the teacher’s response changes performance of the resulting machine learning model on the learning task. In this framework, learning good question-asking strategies corresponds to asking sequences of questions that maximize the cumulative (discounted) reward, and hence quickly lead to effective classifiers. Empirical analysis across three domains shows that learned question-asking strategies expedite classifier training by asking appropriate questions at different points in the learning process. The approach allows learning classifiers from a blend of strategies, including learning from observations, explanations and clarifications.
BibTeX
@inproceedings{srivastava2019learning,
  author = {Shashank Srivastava and Igor Labutov and Tom M. Mitchell},
  title = {{Learning to Ask for Conversational Machine Learning}},
  booktitle = {Conference on Empirical Methods in Natural Language Processing},
  year = {2019},
  url = {https://openreview.net/forum?id=cQq54DKV0b},
}

Loading related content...

Figure