Transferable Contextual Bandit for Cross-Domain Recommendation

Bo Liu, Ying Wei, Yu Zhang, Zhixian Yan, Qiang Yang

Abstract

Recommendation systems often struggle with the "cold-start" problem, where a lack of historical data in a new domain leads to poor initial performance. While contextual bandits address the exploration-exploitation trade-off, they typically require extensive trials to converge. This paper introduces the Transferable Contextual Bandit (TCB), a framework that leverages knowledge from a data-rich source domain to accelerate learning in a target domain. By learning a translation mapping between heterogeneous feature spaces and accounting for dual uncertainties in both user preferences and domain translation, TCB achieves sub-linear regret and significantly outperforms traditional bandit and transfer learning baselines in cold-start scenarios.

Introduction

Modern recommendation systems (RecSys) face a dual challenge: the exploitation-exploration dilemma and the cold-start problem. Contextual bandits are effective at balancing known user preferences with the need to discover new ones, but their performance is limited when starting from scratch. Conversely, cross-domain recommendation methods can "warm-start" a system using data from established domains, but they often lack the adaptive, iterative learning capabilities of bandits.

TCB bridges this gap by combining the adaptive nature of contextual bandits with the power of cross-domain transfer learning. It is designed to handle "heterogeneous" transfer, where the source and target domains have different feature spaces (e.g., mapping app installation history to article click predictions).

The TCB Framework

The core of TCB is its ability to "translate" knowledge across domains while managing uncertainty.

The TCB framework architecture for cross-domain recommendation.

Key Components:

  • Translation Matrix ($U_*$): TCB learns a mapping to align the target domain's context with the source domain's feature space. This allows the model to utilize rich source-domain data even when feature dimensions differ.
  • Dual-Uncertainty Exploration: Unlike standard bandits that only track uncertainty in user preferences ($\theta$), TCB tracks uncertainty in both $\theta$ and the translation matrix ($U$).
  • Upper Confidence Bound (UCB) Strategy: Recommendations are selected based on a score that combines predicted reward (exploitation), reward uncertainty (exploration), and transfer uncertainty (transfer exploration).

The system uses an alternating minimization strategy to optimize the non-jointly convex loss function, iteratively updating the translation matrix and the reward parameters.

Experimental Evaluation

The authors evaluated TCB across synthetic and real-world datasets, including the Cheetah Mobile dataset (App to Article recommendation) and the Delicious bookmarking dataset.

Key Results

Experimental results demonstrate that TCB effectively solves the cold-start problem by achieving faster convergence and lower regret than existing methods.

Cumulative regret comparison across different algorithms.
  • Superior Regret Bounds: As shown in Figure 2, TCB maintains significantly lower cumulative regret compared to standard LinUCB and pure transfer methods like HTL. The authors prove that TCB achieves a regret bound of $O(\sqrt{N_t} \log N_t)$.
  • Cold-Start Performance: In the critical early stages (first 500–1,000 steps), TCB provides more accurate recommendations than non-transfer baselines like LinUCB and hLinUCB.
  • Long-term Adaptability: While pure transfer methods (HTL) may start strong, they often plateau. TCB’s exploration mechanism allows it to refine its translation model over time, eventually outperforming static transfer approaches.

Conclusion

TCB represents a significant step forward in cross-domain recommendation. By transferring the underlying linear reward structure rather than just confidence bounds, it handles heterogeneous domains effectively. While it faces challenges regarding computational overhead and dependency on correspondence data sparsity, its theoretical soundness and empirical performance make it a robust solution for cold-start recommendation scenarios.

Summary updated

Abstract

Traditional recommendation systems (RecSys) suffer from two problems: the exploitation-exploration dilemma and the cold-start problem. One solution to solving the exploitation-exploration dilemma is the contextual bandit policy, which adaptively exploits and explores user interests. As a result, the contextual bandit policy achieves increased rewards in the long run. The contextual bandit policy, however, may cause the system to explore more than needed in the cold-start situations, which can lead to worse short-term rewards. Cross-domain RecSys methods adopt transfer learning to leverage prior knowledge in a source RecSys domain to jump start the cold-start target RecSys. To solve the two problems together, in this paper, we propose the first applicable transferable contextual bandit (TCB) policy for the cross-domain recommendation. TCB not only benefits the exploitation but also accelerates the exploration in the target RecSys. TCB's exploration, in turn, helps to learn how to transfer between different domains. TCB is a general algorithm for both homogeneous and heterogeneous domains. We perform both theoretical regret analysis and empirical experiments. The empirical results show that TCB outperforms the state-of-the-art algorithms over time.
BibTeX
@inproceedings{liu2018transferable,
  author = {Bo Liu and Ying Wei and Yu Zhang and Zhixian Yan and Qiang Yang},
  title = {{Transferable Contextual Bandit for Cross-Domain Recommendation}},
  booktitle = {AAAI Conference on Artificial Intelligence},
  year = {2018},
  doi = {10.1609/aaai.v32i1.11699},
}
Figure