Deep Learning from Logged Interventions

Thorsten Joachims

Abstract

Every time a system places an ad, presents a search ranking, or makes a recommendation, we can think about this as an intervention for which we can observe the user's response (e.g. click, dwell time, purchase). Such logged intervention data is one of the most plentiful types of data available, as it can be recorded from a variety of systems (e.g., search engines, recommender systems, ad placement) at little cost. However, this data provides only partial-information feedback -- aka "bandit feedback" -- limited to the particular intervention chosen by the system. We don't get to see how the user would have responded, if we had chosen a different intervention. This makes learning from logged bandit feedback substantially different from conventional supervised learning, where "correct" predictions together with a loss function provide full-information feedback. It is also different from online learning in the bandit setting, since the algorithm does not assume interactive control of the interventions. In this talk, I will explore learning methods for batch learning from logged bandit feedback (BLBF). Following the inductive principle of Counterfactual Risk Minimization for BLBF, this talk presents an approach to training linear models and deep networks from propensity-scored bandit feedback.
BibTeX
@inproceedings{joachims2018deep,
  author = {Thorsten Joachims},
  title = {{Deep Learning from Logged Interventions}},
  booktitle = {DLRS@RecSys},
  year = {2018},
  doi = {10.1145/3270323.3270324},
}

Loading related content...

Figure