Introduction
In large-scale recommender systems, the "matching" phase serves as the critical first filter, retrieving a relevant subset of items from a corpus of millions for subsequent ranking. For micro-video platforms like TikTok or WeChat Channels, this task is particularly demanding due to the high volume of content, the diversity of user interests, and the rapid temporal shifts in consumption patterns.
A primary challenge in multi-interest modeling is "interest collapse," where distinct user preferences (e.g., "travel" vs. "pets") are merged into a single, non-descriptive embedding. Furthermore, existing models often fail to capture how specific interests evolve independently over time.
The OPAL Framework
To address these challenges, we introduce OPAL (Orthogonal category and Personalized interest Activated Lever). OPAL structures the embedding space to ensure interest disentanglement through learnable orthogonal hyper-categories and a specialized training regime.
Orthogonal Hyper-category Embeddings
Rather than allowing interest clusters to form randomly—which often leads to redundancy—OPAL learns a set of $k$ global "hyper-category" vectors. To ensure these categories represent distinct interests, the model enforces an orthogonal constraint loss. By treating these categories as unit orthogonal basis vectors in the multidimensional embedding space, OPAL mathematically guarantees heterogeneity between the tracked interests.
Two-Stage Training Strategy
The training process for OPAL is divided into two phases to optimize both broad preference capture and fine-grained temporal modeling.
- Pre-training (Soft Interests): Initially, each video interaction is allowed to contribute to multiple interests based on its cosine similarity to the hyper-categories. This "soft" assignment builds a broad understanding of user preferences and prevents the model from getting stuck in poor local optima early in training.
- Fine-tuning (Hard Interests & Evolution): The model then transitions to "hard" assignments, where each video is routed exclusively to its most relevant interest category. This creates disjoint sub-sequences for each interest. OPAL then employs a Gated Recurrent Unit (GRU) on these sub-sequences to capture how that specific interest has evolved over time.
Future-Item Prediction
Standard models often rely on "next-item prediction," which can be too restrictive for micro-videos where a user might be interested in several videos appearing in their subsequent feed. OPAL formulates the problem as a future-item prediction task, treating any video in the user's subsequent interaction window as a positive sample, providing a more robust training signal.
Experimental Evaluation
OPAL was evaluated against six state-of-the-art competitors (including MIND and ComiRec) on two massive, real-world datasets: WeChat-Channels and MX-TakaTak.
Key Results
- Accuracy: On the WeChat-Channels dataset, OPAL improved Recall@50 by 17.23% and HitRate@50 by 8.91% over the next best model (ComiRec-SA).
- Diversity: Using Shifted Positive Pointwise Mutual Information (SPPMI) to measure correlation, the results showed that OPAL's multi-interest approach significantly reduces correlation between recommended items compared to single-interest models, leading to a more diverse user feed.
- Optimal Interest Density: Performance was found to peak at specific values of $k$ (e.g., $k=4$ for WeChat), proving that while multi-interest modeling is superior, the number of interests must be tuned to the specific dataset.
Conclusion
OPAL demonstrates that enforcing orthogonality in the embedding space and utilizing a soft-to-hard training transition significantly improves both the accuracy and diversity of micro-video matching. Future work will explore dynamically determining the number of interests per user and integrating explicit metadata to further refine the learned hyper-categories.
Summary updated