Joint User Modeling Across Aligned Heterogeneous Sites Using Neural Networks

Xuezhi Cao, Yong YuEcml/pkdd 2017 · DOI

TL;DR

This paper introduces JUN, a novel neural network framework for joint user modeling across heterogeneous online sites. JUN effectively addresses the cold-start problem and improves recommendation quality by leveraging user actions from multiple aligned sites, overcoming limitations of traditional methods that require perfect alignment and struggle with diverse data types.

What Problem Does This Paper Address?

Personalized recommender systems are ubiquitous, but their effectiveness is often hampered by the "cold-start" problem, where new users with limited interaction history are difficult to recommend for. Traditional systems typically model user preferences within a single site, failing to utilize valuable information from other platforms a user might interact with. While cross-domain and joint user modeling approaches exist, they often struggle with:

  • Heterogeneous Actions: Different sites have vastly different types of user interactions (e.g., movie ratings vs. microblog posts).
  • Alignment Requirements: Many methods assume perfect alignment between user sets across sites, which is unrealistic.
  • Hard Constraints: Existing methods often use rigid rules to combine preferences from different sites, which don't reflect the nuanced nature of individual user behavior.

This paper aims to bridge this gap by proposing a flexible and powerful framework that can integrate user data from diverse sites, even with imperfect alignment, to build more robust and accurate user models.

Methods

The core of the proposed solution is JUN (Joint User Modeling), a neural network-based framework designed for joint user modeling across heterogeneous sites. JUN has two main components:

  1. Site-Specific Preference Models: For each site (e.g., item-based like movie ratings, or text-based like microblogs), a dedicated neural network is used to capture user preferences. The paper specifically utilizes Product-based Neural Networks (PNNs) for this, which are adept at modeling interactions and can handle various data types.
  2. Cross-Site Preference Consistency Model: This component is the key to transferring knowledge between sites. Instead of hard constraints, JUN uses a neural network to learn how user preferences are consistent across aligned sites. This is achieved by:
  3. Shared User Embeddings: User representations are learned jointly across sites.
  4. Fine-tuning: An auxiliary neural network is trained to predict alignment between users from different sites. This network's gradients are then backpropagated to fine-tune the shared user embeddings, effectively teaching them to represent both site-specific preferences and cross-site consistency. This approach is inspired by fine-tuning techniques used for word embeddings.

JUN's design emphasizes generality, allowing it to incorporate various site-specific models and handle different types of alignment (partial, score-based).

Datasets

The experiments were conducted using a dataset collected from two distinct online platforms:

  • Douban: An item-based site (movie ratings).
  • Weibo: A text-based site (microblog posts).

This dataset was previously used in joint user modeling research and was filtered to include 27,814 users with sufficient activity. The authors note that this pairing of item-based and text-based actions serves as a representative setting for heterogeneous scenarios.

Key Insights and Ideas

  • Neural Networks for Heterogeneity: The paper strongly advocates for neural networks as the go-to tool for handling heterogeneous data in user modeling, citing their ability to capture complex, non-linear relationships.
  • Fine-tuning for Soft Coupling: Moving away from rigid, "hard" constraints for cross-site preference integration is crucial. Fine-tuning user embeddings based on alignment signals allows for a more adaptive and realistic transfer of knowledge.
  • Generality is Key: JUN is designed to be a flexible framework, not tied to specific site-specific models or perfect alignment. This makes it more applicable to real-world scenarios where data is messy and diverse.
  • Action Independence is Valid: Surprisingly, the analysis showed that user action counts in different sites are largely independent. This validates the premise that combining data from multiple sites is indeed beneficial for enrichment, as one site's activity doesn't fully predict another's.

Key Results

JUN demonstrated significant improvements over existing methods:

  • Overall Performance: For the item-based Douban site, JUN achieved a relative AUC improvement of 2.96% over the best existing cross-site approach. For the text-based Weibo site, the improvement was 2.37%.
  • Cold-Start Scenario: JUN was particularly effective in cold-start situations. It achieved relative AUC improvements of 5.77% for Douban and 13.54% for Weibo, highlighting its ability to leverage cross-site information for users with few interactions.
  • Robustness: The performance drop of JUN when reducing the training data ratio was smaller than for existing models, indicating its robustness.
  • Alignment Flexibility: Experiments showed that JUN can effectively leverage partial alignment and score-based alignment, not just perfect alignment, further enhancing its practical applicability.

Limitations and Future Work

  • Alignment Requirement: While JUN is more flexible than previous methods, it still requires some form of alignment information (even if partial or score-based) to function.
  • Dataset Limitation: The experiments were conducted on only two sites (Douban and Weibo). The authors suggest extending JUN to more than two sites.
  • Future Directions: The authors propose integrating social relations from aligned sites into JUN. They also express interest in jointly conducting network alignment and user modeling, as these tasks are highly related.

Summary updated

BibTeX
@inproceedings{cao2017joint,
  author = {Xuezhi Cao and Yong Yu},
  title = {{Joint User Modeling Across Aligned Heterogeneous Sites Using Neural Networks}},
  booktitle = {Ecml/pkdd},
  year = {2017},
  doi = {10.1007/978-3-319-71249-9\_48},
}
Figure