Abstract
Recommendation systems often struggle with the "cold-start" problem, where a lack of historical data in a new domain leads to poor initial performance. While contextual bandits address the exploration-exploitation trade-off, they typically require extensive trials to converge. This paper introduces the Transferable Contextual Bandit (TCB), a framework that leverages knowledge from a data-rich source domain to accelerate learning in a target domain. By learning a translation mapping between heterogeneous feature spaces and accounting for dual uncertainties in both user preferences and domain translation, TCB achieves sub-linear regret and significantly outperforms traditional bandit and transfer learning baselines in cold-start scenarios.
Introduction
Modern recommendation systems (RecSys) face a dual challenge: the exploitation-exploration dilemma and the cold-start problem. Contextual bandits are effective at balancing known user preferences with the need to discover new ones, but their performance is limited when starting from scratch. Conversely, cross-domain recommendation methods can "warm-start" a system using data from established domains, but they often lack the adaptive, iterative learning capabilities of bandits.
TCB bridges this gap by combining the adaptive nature of contextual bandits with the power of cross-domain transfer learning. It is designed to handle "heterogeneous" transfer, where the source and target domains have different feature spaces (e.g., mapping app installation history to article click predictions).
The TCB Framework
The core of TCB is its ability to "translate" knowledge across domains while managing uncertainty.
Key Components:
- Translation Matrix ($U_*$): TCB learns a mapping to align the target domain's context with the source domain's feature space. This allows the model to utilize rich source-domain data even when feature dimensions differ.
- Dual-Uncertainty Exploration: Unlike standard bandits that only track uncertainty in user preferences ($\theta$), TCB tracks uncertainty in both $\theta$ and the translation matrix ($U$).
- Upper Confidence Bound (UCB) Strategy: Recommendations are selected based on a score that combines predicted reward (exploitation), reward uncertainty (exploration), and transfer uncertainty (transfer exploration).
The system uses an alternating minimization strategy to optimize the non-jointly convex loss function, iteratively updating the translation matrix and the reward parameters.
Experimental Evaluation
The authors evaluated TCB across synthetic and real-world datasets, including the Cheetah Mobile dataset (App to Article recommendation) and the Delicious bookmarking dataset.
Key Results
Experimental results demonstrate that TCB effectively solves the cold-start problem by achieving faster convergence and lower regret than existing methods.
- Superior Regret Bounds: As shown in Figure 2, TCB maintains significantly lower cumulative regret compared to standard LinUCB and pure transfer methods like HTL. The authors prove that TCB achieves a regret bound of $O(\sqrt{N_t} \log N_t)$.
- Cold-Start Performance: In the critical early stages (first 500–1,000 steps), TCB provides more accurate recommendations than non-transfer baselines like LinUCB and hLinUCB.
- Long-term Adaptability: While pure transfer methods (HTL) may start strong, they often plateau. TCB’s exploration mechanism allows it to refine its translation model over time, eventually outperforming static transfer approaches.
Conclusion
TCB represents a significant step forward in cross-domain recommendation. By transferring the underlying linear reward structure rather than just confidence bounds, it handles heterogeneous domains effectively. While it faces challenges regarding computational overhead and dependency on correspondence data sparsity, its theoretical soundness and empirical performance make it a robust solution for cold-start recommendation scenarios.
Summary updated