Introduction
In One-Class Collaborative Filtering (OCCF), models must learn user preferences from implicit feedback, such as clicks or views, where only positive signals are observed. Researchers have developed numerous learning objectives (loss functions) to address this, including Bayesian Personalized Ranking (BPR), Collaborative Metric Learning (CML), and Binary Cross-Entropy (BCE). However, no single objective consistently outperforms others across all datasets. This paper introduces ConCF (Consensus learning for OCCF), a framework that leverages the complementary strengths of multiple heterogeneous objectives to improve recommendation accuracy.
Methodology
ConCF employs a multi-branch architecture during training to capture diverse facets of user behavior. The framework consists of a shared user-item encoder and multiple auxiliary heads, each optimized with a different learning objective (e.g., BPR, CML, BCE, MSE, and Multinomial).
Ranking-Based Consensus
Because different loss functions produce scores in varying ranges, ConCF converts these scores into relative rankings to create a "universal language" for knowledge exchange. A central Consensus module aggregates these rankings. To ensure the consensus is reliable, the model utilizes temporal consistency: it identifies stable predictions across training epochs as more likely to be correct, weighting the consensus toward these reliable signals.
Collaborative Evolution and Inference
During training, each head is pulled toward the collective consensus while simultaneously satisfying its specific objective. To balance these potentially conflicting gradients, the framework uses dynamic gradient normalization (GradNorm). Crucially, the auxiliary heads are discarded after training, resulting in a standard single-head model that maintains inference efficiency while benefiting from multi-objective knowledge.
Empirical Motivation and Results
The authors demonstrate that different objectives are indeed complementary using the Pairwise Exclusive-hit Ratio (PER). Even when one objective is statistically superior, other objectives often correctly predict items that the "better" model misses.
Key findings from evaluations on CiteULike, Ciao, and Foursquare include:
- Significant Accuracy Gains: ConCF consistently outperformed standalone objectives, improving Recall@20 by approximately 25% on the CiteULike dataset compared to the best single-objective baseline.
- Superiority over Ensembles: The framework outperformed traditional model ensembles, suggesting that the interaction between objectives during training is more effective than post-hoc combination.
- Robustness for Sparse Users: The consensus approach proved particularly beneficial for "cold" users with few interactions, where the shared knowledge helps fill data gaps.
Conclusion
ConCF provides a robust framework for improving OCCF by harmonizing heterogeneous learning objectives. By identifying stable, consistent predictions across multiple heads, the model learns a more comprehensive representation of user preferences. Future work will explore extending this consensus mechanism to more complex architectures like Graph Neural Networks and Transformers.
Summary updated