SeeDRec: Sememe-based Diffusion for Sequential Recommendation

Haokai Ma, Ruobing Xie, Lei Meng, Yimeng Yang, Xingwu Sun, Zhanhui Kang

Abstract

Diffusion Models (DM) have emerged as powerful tools for generating high-quality distributions in recommendation systems. However, existing item-level diffusion approaches face significant challenges regarding computational scalability, cross-domain transferability, and semantic awareness. This paper introduces SeeDRec, a novel framework that performs diffusion on sememes—the smallest semantic units of user interest—rather than massive item catalogs. By mapping items to a compact, universal sememe space, SeeDRec enables efficient interest modeling and superior performance in sequential recommendation tasks, particularly in cross-domain and few-shot scenarios.

Introduction

Sequential Recommendation (SR) aims to predict a user's next interaction based on their historical behavior. While Diffusion Models have shown promise, current state-of-the-art methods typically perform diffusion on item indices (IDs). This approach suffers from three major flaws:

  1. Poor Scalability: Computational complexity explodes as item catalogs grow into the millions.
  2. Lack of Transferability: Item IDs are specific to a single dataset, preventing easy transfer between domains (e.g., from Books to Electronics).
  3. Semantic Blindness: ID-based models often ignore the underlying semantic relationships between items, such as shared genres or themes.

SeeDRec addresses these issues by shifting the diffusion paradigm to the sememe level, treating user interests as a "blooming flower" cultivated from a "seed" of original behaviors.

Methods

The SeeDRec framework consists of two primary components designed to capture and utilize atomic user interests:

  • Sememe-to-Interest Diffusion Model (S2IDM): Instead of diffusing item IDs, SeeDRec utilizes sememes (e.g., "Cartoon," "Funny," "Sensor"). By mapping items to a shared sememe space, the diffusion process occurs in a much lower-dimensional environment ($|S| \ll |I|$). A frequency-time aware reweighting strategy is employed to prioritize recent and significant interests before the noising process begins.
  • Interest-aware Prompt-enhanced (IPE) Strategy: The S2IDM generates a "generalized interest distribution," which IPE converts into continuous prompts. These prompts are injected into a standard sequence encoder (like SASRec) to provide a "semantic compass," helping the model better understand the user's next likely move.
Detailed architecture of the SeeDRec framework, illustrating the transition from item IDs to sememes and the subsequent diffusion and prompting processes.

Results

The researchers evaluated SeeDRec on 13 real-world datasets, including five standard SR datasets (PixelRec and four Amazon categories: Home, Electronic, CD, Toy) and several cross-domain pairs.

Key Performance Metrics

  • State-of-the-Art Performance: SeeDRec consistently outperformed base models and existing DM-based models. On the PixelRec dataset, it improved NDCG@10 by over 20%.
  • Cross-Domain Superiority: In transfer tasks, SeeDRec showed significant gains because it could map source domain interests to the target domain via the shared sememe space.
  • Few-Shot Robustness: The model was particularly effective for "cold-start" users. In scenarios with very few interactions, SeeDRec's ability to generalize interests via diffusion led to a 41% improvement in NDCG@10 on the PixelRec dataset.
Performance comparison of SeeDRec against various baselines across five sequential recommendation datasets, measuring NDCG@10 and HR@10.

Conclusion

SeeDRec represents a significant advancement in recommendation system design by leveraging atomic interests (sememes) within a diffusion framework. This approach not only improves efficiency and performance but also enables robust zero-shot transfer across domains. Future work will focus on refining the definition of minimal interest units and exploring more complex semantic correlations during the diffusion process to capture even more nuanced user patterns.

Summary updated

Abstract

Inspired by the power of Diffusion Models (DM) verified in various fields, some pioneering works have started to explore DM in recommendation. However, these prevailing endeavors commonly implement diffusion on item indices, leading to the increasing time complexity, the lack of transferability, and the inability to fully harness item semantic information. To tackle these challenges, we propose SeeDRec, a sememe-based diffusion framework for sequential recommendation (SR). Specifically, inspired by the notion of sememe in NLP, SeeDRec first defines a similar concept of recommendation sememe to represent the minimal interest unit and upgrades the specific diffusion objective from the item level to the sememe level. With the Sememe-to-Interest Diffusion Model (S2IDM), SeeDRec can accurately capture the user's diffused interest distribution learned from both local interest evolution and global interest generalization while maintaining low computational costs. Subsequently, an Interest-aware Prompt-enhanced (IPE) strategy is proposed to better guide each user's sequential behavior modeling via the learned user interest distribution. Extensive experiments on nine SR datasets and four cross-domain SR datasets verify its effectiveness and universality. The code is available in https://github.com/hulkima/SeeDRec.
BibTeX
@inproceedings{ma2024seedrec,
  author = {Haokai Ma and Ruobing Xie and Lei Meng and Yimeng Yang and Xingwu Sun and Zhanhui Kang},
  title = {{SeeDRec: Sememe-based Diffusion for Sequential Recommendation}},
  booktitle = {International Joint Conference on Artificial Intelligence},
  year = {2024},
  url = {https://openreview.net/forum?id=xSreKXf622},
}
Figure