TL;DR
This paper introduces the first online cross-network recommendation system using a novel multi-layered LSTM architecture. It addresses limitations of existing offline models by incorporating attention mechanisms, higher-order interactions, and time-aware gates to capture complex, dynamic user preferences across different social platforms for more accurate and diverse recommendations.
What Problem Does This Paper Address?
In today's digital landscape, users interact across multiple online social networks (OSNs) like Twitter, Facebook, and YouTube. While individual OSNs offer insights into user preferences, they often suffer from data sparsity, meaning there aren't enough interactions to build a comprehensive user profile. Cross-network recommender systems aim to overcome this by leveraging data from multiple sources to create richer user profiles and improve recommendations in a target network. However, existing cross-network solutions face two major challenges:
- Inability to Capture Complex Non-linear Relationships: Many current models rely on linear approaches (like Matrix Factorization) which fail to capture the intricate, non-linear patterns in user behavior.
- Offline Nature: User preferences are dynamic and constantly evolving. Existing systems are typically offline, meaning they aren't updated in real-time with new interactions, leading to stale recommendations. This is particularly problematic in cross-network scenarios where user interests can shift rapidly across different platforms.
Methods
The authors propose a novel multi-layered Long Short-Term Memory (LSTM) network designed for online cross-network recommendations. Their model extends the standard LSTM with three key innovations:
- Cross-network Topical Layer: To handle the heterogeneity of data from different sources (e.g., tweets vs. YouTube likes), they use a topic modeling approach (Twitter-LDA) to represent interactions in a common topical space. This allows for integration of diverse user activities.
- Higher-Order Interaction Layer: To combat data sparsity, this layer goes beyond simple feature interactions. Inspired by Factorization Machines (FMs), it computes second-order interactions between embeddings of user and item features, effectively creating richer feature representations.
- Attention Mechanism: This mechanism allows the LSTM to selectively focus on relevant parts of a user's historical interactions, reducing the burden on the LSTM cell to encode the entire history. It weighs past preferences based on their relevance to the current context.
- Time-Aware Input and Forget Gates: Standard LSTMs assume uniform time intervals between data points. This model introduces specialized gates that account for irregular time gaps between user interactions, recognizing that smaller gaps might indicate more stable preferences, while larger gaps might signal significant shifts.
- Modified LSTM Cell: The core LSTM cell is augmented with these attention and time-aware gates to process the combined information from the cross-network topical layer, higher-order interactions, and temporal dynamics.
- Prediction Layer: A final layer uses a hyperbolic tangent activation and softmax function to predict the probability of a user interacting with each item, generating a ranked list of recommendations.
The model is trained offline initially and then updated online with incoming interactions.
Datasets
The authors used data extracted from two public datasets and scraped additional timestamped interactions over a two-year period (March 2015 to February 2017).
- Source Networks: Twitter and Google Plus.
- Target Network: YouTube (specifically, liked videos or videos added to playlists).
To handle data sparsity and cold-start issues, they filtered users with fewer than 10 interactions on Twitter, 5 on Google Plus, and users/videos with fewer than 10 interactions on YouTube. The final dataset comprised 2,536 users and 13,829 YouTube videos, with an overall user-video matrix sparsity of 99.64%. The data was split into training (70%), validation (10%), and testing (20%) sets, simulating a dynamic data stream for online evaluation.
Key Insights and Ideas
- The Power of Online, Cross-Network Learning: The paper convincingly demonstrates that combining data from multiple sources and updating recommendations in real-time is crucial for capturing evolving user preferences and achieving superior performance.
- Beyond Linearity is Essential: The significant performance gains over linear models (like MF) highlight that user behavior is inherently complex and non-linear, necessitating more sophisticated modeling techniques.
- Attention as a Memory Aid: The attention mechanism acts as a smart way to manage the vast amount of historical data, allowing the model to recall relevant past experiences without being overwhelmed, which is key for maintaining accuracy over time.
- Time Matters (Even Irregularly): Explicitly modeling the time intervals between interactions is not just an add-on; it's a critical component for understanding the dynamics of preference shifts.
- Sparsity is a Persistent Challenge, but Solvable: The higher-order interaction layer effectively tackles data sparsity, showing that feature engineering and interaction modeling can significantly boost performance in sparse recommendation scenarios.
Key Results
The proposed model consistently outperformed various baselines, including:
- Accuracy (HR and NDCG): The proposed model achieved the highest Hit Ratio (HR) and Normalized Discounted Cumulative Gain (NDCG) across test interactions. For instance, it consistently outperformed CLSTM (a strong non-linear cross-network baseline) and Time-LSTM (a strong single-network online model).
- Diversity and Novelty: Beyond accuracy, the model showed significant improvements in diversity (7.7%) and novelty (12.8%) compared to the closest baseline (CLSTM), indicating more satisfying and less predictable recommendations.
- Robustness to Sparsity: Experiments showed the proposed model maintained higher accuracy and was more robust against sparsity compared to a version without the higher-order interaction layer.
- Online Performance: The model demonstrated its ability to maintain achieved accuracy over time, with a slower decline compared to other models when new interactions were streamed in.
Limitations and Future Work
- Limitations:
- The paper acknowledges that the contribution of time-aware input/forget gates to model accuracy was comparatively lower than other variations, suggesting potential for improvement with more complex temporal functions.
- The dimensionality of embedding vectors (k) needs careful tuning; higher values increase representation power but risk overfitting, with the optimal value depending on dataset size.
- Future Work:
- The authors plan to extend the model to incorporate social influences, such as how changes in friends' preferences might impact a user's own recommendations.
- Exploring more complex temporal functions for the time-aware gates is suggested.
- Investigating learning the similarity function for attention scores (currently a cosine similarity) using a feed-forward neural network is also proposed.
Summary updated