SeeDRec: Sememe-based Diffusion for Sequential Recommendation

Haokai Ma, Ruobing Xie, Lei Meng, Yimeng Yang, Xingwu Sun, Zhanhui Kang

Abstract

Sequential recommendation systems (SRS) traditionally rely on item IDs, which limits scalability and cross-domain transferability. We present SeeDRec, a novel framework that leverages Diffusion Models to denoise user interest at the "sememe" level—the fundamental, atomic units of semantic interest. By decomposing items into sememes, SeeDRec reduces the diffusion target space and enables zero-shot transfer across domains. Our Sememe-to-Interest Diffusion Model (S2IDM) captures global interest correlations, while the Interest-aware Prompt-enhanced (IPE) strategy bridges semantic distributions with sequence encoders. Experiments on 13 real-world datasets demonstrate that SeeDRec significantly outperforms state-of-the-art baselines, particularly in data-sparse few-shot scenarios.

Introduction

Diffusion Models (DMs) have demonstrated remarkable capabilities in generating complex distributions in computer vision and audio processing. However, their application to Recommender Systems (RS) has primarily focused on item indices. This ID-based approach faces three critical challenges: poor scalability as item catalogs grow, zero transferability between datasets with disjoint ID spaces, and semantic blindness to the underlying relationships between items. To overcome these hurdles, we propose shifting the diffusion target from specific item IDs to universal semantic units called sememes.

Methodology: The SeeDRec Framework

SeeDRec introduces a paradigm shift by representing user history as a distribution over a sememe vocabulary rather than a sequence of item IDs. This approach consists of three core components:

  1. Recommendation Sememes: Items are decomposed into a set of sememes (e.g., "4K," "Cheap," "Type-C"), reducing the diffusion target space from the size of the item catalog ($|I|$) to a much smaller sememe vocabulary ($|S|$).
  2. Sememe-to-Interest Diffusion Model (S2IDM): Based on Denoising Diffusion Probabilistic Models (DDPM), S2IDM learns to denoise a user's historical sememe distribution. This process captures both local behavioral patterns and global semantic correlations, producing a Generalized Interest Distribution.
  3. Interest-aware Prompt-enhanced (IPE) Strategy: To integrate the discrete sememe distribution with continuous sequence encoders (e.g., SASRec), SeeDRec employs a prompt-learning paradigm. The generalized interest distribution is transformed into continuous prompts that guide the encoder to focus on the most relevant aspects of a user's history.
The SeeDRec architecture, illustrating the flow from item IDs to sememe distributions, the S2IDM denoising process, and the prompt-enhanced sequence encoding.

Experiments and Evaluation

We evaluated SeeDRec on 13 real-world datasets, including PixelRec and various Amazon categories (Home, Electronic, CD, Toy). For datasets lacking explicit taxonomies, sememes were extracted from item titles using natural language processing techniques.

Performance and Scalability

SeeDRec consistently outperformed strong baselines such as SASRec, CL4SRec, and PDRec. On the PixelRec dataset, SeeDRec achieved a 21% improvement in NDCG@10 over SASRec. Furthermore, the sememe-based approach significantly improves efficiency; on the "CD" dataset, SeeDRec's diffusion component required only 0.92M parameters, compared to over 200M for traditional ID-based diffusion models.

Performance comparison of SeeDRec against backbone models and ablation variants across five major datasets (Home, Electronic, Toy, CD, Pixel).

Few-Shot and Cross-Domain Transfer

The model demonstrates exceptional robustness in data-sparse scenarios. In few-shot tests involving users with minimal interactions, SeeDRec improved NDCG@10 by up to 41%. Additionally, because sememes are domain-agnostic (e.g., "Action" or "Electronic" maintain meaning across platforms), the S2IDM core can be transferred between domains without retraining, facilitating effective cross-domain recommendation.

Conclusion

SeeDRec represents a significant advancement in applying generative diffusion models to recommendation systems. By abstracting user interest into sememes, the framework achieves superior scalability, transferability, and performance. Future work will explore the automated learning of latent sememes and more complex interactions between semantic units to further refine user persona capture.

Summary updated

BibTeX
@inproceedings{ma2024seedrec,
  author = {Haokai Ma and Ruobing Xie and Lei Meng and Yimeng Yang and Xingwu Sun and Zhanhui Kang},
  title = {{SeeDRec: Sememe-based Diffusion for Sequential Recommendation}},
  booktitle = {International Joint Conference on Artificial Intelligence},
  year = {2024},
  url = {https://openreview.net/forum?id=xSreKXf622},
}
Figure