Sub-Interest-Aware Representation Uniformity for Recommender System

Ruijia Ma, Yahong Lian, Chunyao SongAAAI 2025 · DOI

TL;DR

This paper introduces SIURec, a novel recommender system approach that tackles the challenge of balancing user representation "uniformity" (dissimilarity among users) and "alignment" (similarity to items they like). SIURec achieves this by optimizing uniformity at each layer of the neural network, capturing finer-grained "sub-interests," and dynamically learning the optimal trade-off between uniformity and alignment, leading to improved recommendation accuracy.

What Problem Does This Paper Address?

Recommender systems are crucial for navigating the vast amount of information available online. A key to their effectiveness lies in how well they represent users and items. For personalized recommendations, it's important that user representations are distinct from each other (uniformity) while also being close to the items they've interacted with (alignment). However, achieving both simultaneously is a difficult balancing act. Previous methods often optimize uniformity only on the final user/item representations, which can lead to unrealistic representations that don't accurately reflect real-world user preferences. For example, forcing too much uniformity on the final embedding might push users with similar interests too far apart, hindering recommendations for shared items. Furthermore, existing approaches often rely on manually tuning the weights between uniformity and alignment, which is suboptimal and lacks flexibility.

Methods

SIURec proposes a two-pronged approach to address these limitations:

  • Sub-Interest-Aware Uniformity: Instead of optimizing uniformity on the final aggregated user/item embeddings, SIURec refines representations at each convolutional layer. It achieves this by first generating a "pseudo-interaction matrix" that captures more nuanced preference scores. Then, it applies uniformity optimization to these "fine-grained sub-interests" at each layer. This layer-wise optimization allows the model to capture more granular user preferences that might be lost in the final aggregation.
  • Automated Weight Adaptation (One-shot Weighting): To overcome the manual tuning of weights between uniformity and alignment, SIURec introduces a mechanism that automatically learns these weights. It leverages the gradients from the uniformity and alignment (specifically, Bayesian Personalized Ranking - BPR loss) tasks. By analyzing the ratio of these gradients, the model dynamically adjusts the importance of each objective. Crucially, this weight calculation is performed only once in the first training epoch ("one-shot") and then fixed, significantly improving training efficiency without sacrificing performance.

Datasets

The authors evaluated SIURec on four publicly available benchmark datasets:

  • MovieLens-1M (ML-1M)
  • Gowalla
  • Amazon-Beauty (Beauty)
  • Amazon-Book (Book)

These datasets represent diverse domains and user interaction patterns, allowing for a comprehensive evaluation of the proposed method.

Key Insights and Ideas

  • Layer-wise uniformity is key: Optimizing uniformity at each layer, rather than just the final output, is crucial for capturing nuanced user "sub-interests" and preventing unrealistic representations.
  • Sub-interests matter more than final embeddings for uniformity: The paper argues that focusing uniformity on intermediate "sub-interest" representations (denoted as Z) is more effective than on the final aggregated embeddings (E), as Z can capture more nuanced preferences.
  • Automated weight tuning is efficient and effective: The "one-shot weighting" approach, which learns optimal weights for different objectives in the first epoch, significantly reduces computational overhead while maintaining or improving performance compared to continuous weight learning.
  • Robustness across datasets: SIURec consistently outperforms existing methods across various datasets, demonstrating its adaptability and generalizability.

Key Results

SIURec consistently outperformed eleven state-of-the-art methods across the four datasets, achieving an average improvement of 4.26% in accuracy. The paper highlights that AU-based methods, which directly optimize ultimate node representations, often lag behind SIURec, suggesting that their approach to uniformity might be less effective or realistic. The ablation study confirmed the effectiveness of each component of SIURec, with the fine-grained sub-interest extraction and layer-wise uniformity optimization modules being particularly impactful. The one-shot weighting mechanism was also shown to be efficient and robust.

Limitations and Future Work

The paper acknowledges that while SIURec demonstrates robustness, the optimal weight allocation for different layers might still vary. Future work could explore more sophisticated methods for dynamically adjusting these weights throughout training if necessary. Additionally, while the paper focuses on uniformity and alignment, exploring other aspects of representation learning in recommender systems could further enhance performance. The authors also suggest that further research into the interplay between different types of sub-interests and their impact on recommendation quality would be beneficial.

Summary updated

BibTeX
@inproceedings{ma2025sub,
  author = {Ruijia Ma and Yahong Lian and Chunyao Song},
  title = {{Sub-Interest-Aware Representation Uniformity for Recommender System}},
  booktitle = {AAAI Conference on Artificial Intelligence},
  year = {2025},
  doi = {10.1609/aaai.v39i12.33345},
}
Figure