Motif-based Prompt Learning for Universal Cross-domain Recommendation

Bowen Hao, Chaoqun Yang, Lei Guo, Junliang Yu, Hongzhi Yin

Introduction

Cross-Domain Recommendation (CDR) is a critical technique for addressing data sparsity and the cold-start problem by transferring knowledge from a source domain to a target domain. Despite its potential, current CDR models face significant challenges: structural blindness, where complex interaction topologies are ignored, and objective mismatch, where the goals of pre-training and recommendation tasks diverge, potentially leading to "negative transfer."

To overcome these hurdles, we introduce MOP (Motif-based Prompt learning). MOP is a universal framework designed to capture higher-order structural patterns, known as motifs, and utilize a "Pre-training & Prompt Tuning" (PPT) paradigm to unify learning objectives across domains.

Methodology

Motif-Based Structural Modeling

A core innovation of MOP is its focus on motifs—sub-graph structures that represent recurring interaction patterns. Unlike simple user-item edges, these higher-order structures are often more consistent across different platforms, making them ideal for cross-domain knowledge transfer.

The three types of motifs proposed in MOP: Butterfly ($\beta$), Random Walk ($\mathcal{R}$), and Triangle ($\mathcal{T}_1, \mathcal{T}_2, \mathcal{T}_3$).

MOP specifically samples three types of motifs:

  • Butterfly ($\beta$): Represents dense, fully connected short-term dependencies.
  • Triangle ($\mathcal{T}$): Captures partially connected relationships, such as those found in social circles.
  • Random Walk ($\mathcal{R}$): Models long-term, sequential dependencies between users and items.

Encoder and Prompt-Based Tuning

The MOP architecture employs a Motif-based Encoder consisting of Hypergraph Convolutions to process multi-node relationships and a Mixture-of-Domain-Experts (MoDE) Transformer to generate domain-shared and domain-specific embeddings.

The "secret sauce" of MOP lies in its Pre-training & Prompt Tuning (PPT) strategy. By unifying Self-Supervised Learning (SSL) and recommendation tasks into a single Motif-based Similarity Learning (MSL) template, MOP bridges the objective gap.

The MOP architecture illustrating the pre-training and prompt-based readout phases, highlighting frozen and updated components.

During the recommendation phase, the backbone model is frozen to preserve general knowledge, while only lightweight "prompt" parameters in the ReadOut function are tuned. This ensures the model adapts to the target domain without losing the structural insights gained during pre-training.

Key Results and Insights

MOP was evaluated across four distinct CDR scenarios using Amazon and movie datasets. It consistently outperformed state-of-the-art (SOTA) baselines like UniCDR and CAT-ART.

  • Performance Gains: In Scenario 1 (Sport), MOP achieved a Hit Ratio (HR@10) of 19.23%, a notable improvement over the SOTA's 18.37%.
  • Cold-Start Success: MOP demonstrated significant effectiveness in inter-domain recommendations, proving that motif-based knowledge is highly transferable even when user history in the target domain is non-existent.
  • Ablation Analysis: The "Butterfly" motif was identified as the most influential component, likely because it captures the most stable and recurring patterns of user-item interactions.

Conclusion

MOP represents a significant advancement in universal cross-domain recommendation. By leveraging higher-order motifs and a unified prompt-tuning paradigm, it successfully addresses the limitations of structural blindness and objective mismatch, providing a robust framework for effective knowledge transfer across diverse recommendation platforms.

Summary updated

Abstract

Cross-Domain Recommendation (CDR) stands as a pivotal technology addressing issues of data sparsity and cold start by transferring general knowledge from the source to the target domain. However, existing CDR models suffer limitations in adaptability across various scenarios due to their inherent complexity. To tackle this challenge, recent advancements introduce universal CDR models that leverage shared embeddings to capture general knowledge across domains and transfer it through "Multi-task Learning'' or "Pre-train, Fine-tune'' paradigms. However, these models often overlook the broader structural topology that spans domains and fail to align training objectives, potentially leading to negative transfer. To address these issues, we propose a motif-based prompt learning framework, MOP, which introducesmotif-based shared embeddings to encapsulate generalized domain knowledge, catering to both intra-domain and inter-domain CDR tasks. Specifically, we devise three typical motifs: butterfly, triangle, and random walk, and encode them through a Motif-based Encoder to obtain motif-based shared embeddings. Moreover, we train MOP under the "Pre-training & Prompt Tuning'' paradigm. By unifying pre-training and recommendation tasks as a common motif-based similarity learning task and integrating adaptable prompt parameters to guide the model in downstream recommendation tasks, MOP excels in transferring domain knowledge effectively. Experimental results on four distinct CDR tasks demonstrate the effectiveness of MOP than the state-of-the-art models.
BibTeX
@inproceedings{hao2023motif,
  author = {Bowen Hao and Chaoqun Yang and Lei Guo and Junliang Yu and Hongzhi Yin},
  title = {{Motif-based Prompt Learning for Universal Cross-domain Recommendation}},
  booktitle = {WSDM},
  year = {2023},
}
Figure