Diversified Interactive Recommendation with Implicit Feedback

Yong Liu, Yingtai Xiao, Qiong Wu, Chunyan Miao, Juyong Zhang, Binqiang Zhao, Haihong Tang

Introduction

Modern recommendation systems are increasingly interactive, learning from user feedback in real-time. However, they frequently encounter two significant challenges: the accuracy-diversity dilemma and implicit feedback bias. Standard algorithms often prioritize immediate clicks (accuracy), leading to repetitive recommendations that trap users in "filter bubbles." Furthermore, real-world data is often implicit (e.g., clicks vs. non-clicks), where a lack of interaction doesn't necessarily imply dislike but could result from a lack of exposure.

This paper introduces DC²B (Diversified Contextual Combinatorial Bandit), a framework designed to balance relevance with variety while mathematically accounting for exposure bias in implicit feedback.

Methodology: The DC²B Framework

The DC²B model treats recommendation as a sequential game within a contextual bandit framework. Its core innovation lies in the integration of three key components:

  1. Determinantal Point Process (DPP): Used to select a "super arm" (a set of items). Unlike greedy selection, DPP considers both the individual quality of items and their mutual dissimilarity, ensuring a diverse slate.
  2. Thompson Sampling with Variational Inference: To learn model parameters from binary click data, the authors use Thompson Sampling. They address the non-conjugate nature of logistic likelihood using Variational Bayesian inference, providing a closed-form approximation for belief updates.
  3. Exposure Modeling: DC²B explicitly models the probability that a set of items is exposed to a user. This allows the system to distinguish between a user's lack of interest and a lack of visibility due to redundancy.

A critical feature of the model is the α parameter, which serves as a control knob for the trade-off between precision and diversity.

Figure 1: Impact of the α parameter on Precision@50 and Diversity. Tuning α allows for a controlled trade-off between accuracy and variety.

Experimental Evaluation

The authors evaluated DC²B against several baselines (LogRank, MMR, ε-Greedy, DPP$^{map}$, C²UCB, and EC-Bandit) using three benchmark datasets: MovieLens-100K, MovieLens-1M, and a large, sparse Anime dataset. High ratings were treated as "clicks," and item embeddings were used as contextual features.

Key Results

DC²B consistently demonstrated superior performance across all datasets, particularly in balancing competing metrics.

Figure 4: Performance comparison across datasets. DC²B (last column) achieves the best F-measure by balancing high precision with robust diversity.
  • Optimal Balance: DC²B achieved the highest F-measure (the harmonic mean of precision and diversity) in nearly all test cases.
  • Performance in Sparse Data: On the Anime dataset, DC²B outperformed the C²UCB baseline by 59.35% in Precision@50, highlighting its effectiveness in data-sparse environments.
  • Stability: Combinatorial methods like DC²B, which update based on the feedback of the entire displayed set, proved more stable than single-item update methods.

Conclusion and Future Directions

DC²B effectively addresses the accuracy-diversity trade-off in interactive recommendation. While the current implementation uses a greedy approximation for DPP and pre-learned embeddings, future research could explore end-to-end representation learning within the bandit framework and more sophisticated DPP kernels to further refine recommendation nuance.

Summary updated

Abstract

Interactive recommender systems that enable the interactions between users and the recommender system have attracted increasing research attention. Previous methods mainly focus on optimizing recommendation accuracy. However, they usually ignore the diversity of the recommendation results, thus usually results in unsatisfying user experiences. In this paper, we propose a novel diversified recommendation model, named Diversified Contextual Combinatorial Bandit (DC2B), for interactive recommendation with users' implicit feedback. Specifically, DC2B employs determinantal point process in the recommendation procedure to promote diversity of the recommendation results. To learn the model parameters, a Thompson sampling-type algorithm based on variational Bayesian inference is proposed. In addition, theoretical regret analysis is also provided to guarantee the performance of DC2B. Extensive experiments on real datasets are performed to demonstrate the effectiveness of the proposed method in balancing the recommendation accuracy and diversity.
BibTeX
@inproceedings{liu2020diversified,
  author = {Yong Liu and Yingtai Xiao and Qiong Wu and Chunyan Miao and Juyong Zhang and Binqiang Zhao and Haihong Tang},
  title = {{Diversified Interactive Recommendation with Implicit Feedback}},
  booktitle = {AAAI Conference on Artificial Intelligence},
  year = {2020},
  doi = {10.1609/aaai.v34i04.5931},
}
Figure