Addressing Cold-Start Problem in Click-Through Rate Prediction via Supervised Diffusion Modeling

Wenqiao Zhu, Lulu Wang, Jun WuAAAI 2025 · DOI

TL;DR

This paper introduces CSDM, a novel diffusion model designed to tackle the "cold-start" problem in Click-Through Rate (CTR) prediction. By leveraging item side information, CSDM generates "warmed-up" embeddings for new items, significantly improving prediction accuracy without altering existing recommendation architectures.

What Problem Does This Paper Address?

Click-Through Rate (CTR) prediction is fundamental to recommendation systems, determining which items users see. The dominant "Embedding & MLP" paradigm struggles with "cold-start" items – those with few or no user interactions. This leads to poorly learned embeddings for these items, hindering their performance. Existing solutions often rely on meta-learning or side information but can be limited by data scarcity or model collapse. This paper aims to address this by creating a more robust way to generate initial embeddings for new items using diffusion models.

Methods

The core of CSDM is a novel diffusion model that creates a transition between an item's ID embedding and its side information. Unlike standard diffusion models that map data to Gaussian noise, CSDM's process is supervised and incorporates side information to enrich the initial ID embeddings. Key aspects include:

  • Supervised Diffusion Process: A diffusion process is designed to gradually incorporate side information into the item ID embedding. This process is supervised by both variational inference and binary cross-entropy objectives.
  • Non-Markovian Property: The diffusion process is designed to be non-Markovian, allowing for the extraction of sub-sequences during generation. This significantly speeds up training without sacrificing performance.
  • Warmed-up Embeddings: The diffusion process generates "warmed-up" embeddings that are then used to replace the initial, poorly learned ID embeddings. This is done without modifying the underlying CTR prediction model architecture.
  • Model-Agnostic: CSDM is designed to be model-agnostic, meaning it can be applied to various backbone CTR prediction models (e.g., DeepFM, Wide & Deep, DCN).

Datasets

The authors evaluated their method on three publicly available datasets:

  • MovieLens-1M
  • Taobao Display Ad Click
  • CIKM 2019 EComm AI

The datasets were split into "cold" and "warm-up" (warm-a, warm-b, warm-c) stages to simulate different levels of item interaction data.

Key Insights and Ideas

  • Diffusion for Embeddings: The paper successfully adapts diffusion models, typically used for image generation, to the domain of embedding generation for CTR prediction. This opens up new avenues for leveraging generative models in recommendation systems.
  • Bridging ID and Side Information: The novel diffusion process effectively bridges the gap between item ID embeddings and item side information, allowing the latter to enrich the former, especially for cold-start items.
  • Non-Markovian Advantage: The non-Markovian nature of their diffusion process is a crucial innovation, enabling faster training through sub-sequence sampling without compromising the quality of generated embeddings.
  • Model Agnosticism: The ability to integrate CSDM with various existing CTR prediction models makes it a versatile solution for addressing the cold-start problem across different recommendation architectures.

Key Results

  • Outperforms Baselines: CSDM consistently outperforms state-of-the-art cold-start recommendation methods across all three datasets and various stages (cold, warm-a, warm-b, warm-c).
  • Significant Improvements: For instance, on the MovieLens-1M dataset, CSDM achieved a 5.62% improvement in AUC in the cold-start phase and a 21.02% improvement in the warm-a phase compared to the DeepFM baseline. Similar significant gains were observed on other datasets.
  • Faster Training: While diffusion models are generally computationally intensive, CSDM's non-Markovian approach with sub-sequence sampling leads to faster training times, as demonstrated in Figure 3.
  • Minimal Inference Overhead: Once embeddings are generated, they are directly used by the backbone model, incurring no additional computational cost during inference.

Limitations and Future Work

  • Computational Cost: Although CSDM speeds up training via sub-sequences, diffusion models inherently require more computational resources than traditional methods. The paper notes that CSDM requires more forward passes than CVAR.
  • Dropout Impact: The effectiveness of dropout in the diffusion model varies across datasets, suggesting potential for further tuning or exploration of regularization techniques.
  • Future Directions: The authors suggest that their model-agnostic approach could be extended to other recommendation tasks beyond CTR prediction. Further research could explore more sophisticated ways to integrate side information or optimize the diffusion process for even greater efficiency.

Summary updated

BibTeX
@inproceedings{zhu2025addressing,
  author = {Wenqiao Zhu and Lulu Wang and Jun Wu},
  title = {{Addressing Cold-Start Problem in Click-Through Rate Prediction via Supervised Diffusion Modeling}},
  booktitle = {AAAI Conference on Artificial Intelligence},
  year = {2025},
  doi = {10.1609/aaai.v39i12.33469},
}
Figure