Abstract
Temporal interaction networks, common in digital ecosystems like e-commerce and social media, involve users and items that evolve continuously. Traditional dynamic embedding methods often suffer from "stale" profiles because they only update representations during active interactions, ignoring the "drift" that occurs during idle time. This paper introduces JODIE (Joint Dynamic User-Item Embeddings), a novel framework that models the continuous trajectory of user and item embeddings. By combining a coupled Recurrent Neural Network (RNN) architecture with a unique projection operator, JODIE predicts future interactions and user state changes with high accuracy. Furthermore, the introduction of the t-Batch algorithm enables training speeds nearly 10x faster than existing sequential methods.
Introduction
In modern digital platforms, user interests and item characteristics are dynamic. A user's preferences today may differ significantly from their preferences a month ago. Most current models only update a user's embedding when an interaction occurs. Consequently, if a user returns after a long hiatus, the system treats them as if no time has passed, failing to account for potential interest drift.
Additionally, standard recommendation models often face computational bottlenecks. Predicting the next interaction typically requires scoring every possible item ($O(N)$ complexity), and training is frequently slowed by the need to process interactions sequentially to maintain temporal dependencies. JODIE addresses these challenges by modeling embedding trajectories, enabling constant-time recommendations, and parallelizing training.
Methodology
The JODIE framework consists of three primary innovations:
1. Coupled Recurrent Neural Networks
JODIE employs two distinct RNNs—one for users and one for items. These networks are "coupled": when an interaction occurs at time $t$, the user’s RNN is updated using the item’s current embedding, and simultaneously, the item’s RNN is updated using the user’s current embedding. This mechanism effectively captures the mutual influence between entities during an interaction.
2. The Projection Operator
The most significant innovation is the projection operator, which estimates a user's state at any given time, even between interactions. Given the time elapsed since the last interaction ($\Delta$), the operator uses a temporal attention vector to "drift" the embedding forward. This allows the model to predict a user's state at the exact moment of a query rather than relying on a stale state from their last active session.
3. t-Batch Algorithm
To overcome the training bottleneck of sequential processing, the authors developed the t-Batch algorithm. It constructs batches of interactions that are independent (no shared users or items within a single batch) but temporally consistent. This allows for massive parallelization on GPUs while preserving the chronological integrity of each entity's interaction history.
Key Insights
- Trajectory Modeling: By treating embeddings as continuous trajectories rather than static points, JODIE captures the evolving "intent" of users during periods of inactivity.
- Direct Embedding Prediction: Instead of calculating scores for all items, JODIE directly outputs the predicted embedding of the next item. This enables the use of Locality Sensitive Hashing (LSH) to find the nearest item in near-constant time, greatly enhancing scalability.
- Hybrid Embeddings: The model integrates static embeddings (representing long-term traits) with dynamic embeddings (representing short-term evolution), balancing inherent preferences with recent behavioral shifts.
Experimental Results
The model was evaluated on four diverse real-world datasets: Reddit, Wikipedia, LastFM, and MOOC.
- Interaction Prediction: JODIE outperformed six state-of-the-art baselines (including DeepCoevolve and RRN) by at least 20% in Mean Reciprocal Rank (MRR).
- State Change Prediction: In predicting rare events like user bans or student dropouts, JODIE improved the Area Under the Curve (AUC) by 12% on average. Notably, it could predict student attrition up to five interactions in advance.
- Training Efficiency: The t-Batch algorithm achieved a 9.2x speedup compared to similar dynamic embedding baselines, allowing models to be trained on millions of interactions in minutes.
Conclusion and Future Work
JODIE represents a significant step forward in modeling temporal interaction networks by accounting for entity drift and enabling scalable, real-time predictions. While highly effective for bipartite networks, future work could extend this approach to account for higher-order graph structures like communities. Other promising directions include trajectory clustering for parameter reduction and generative modeling to predict "gaps" in the market for new item recommendations.
Summary updated