CoT4Rec: Revealing User Preferences Through Chain of Thought for Recommender Systems

Weiqi Yue, Yuyu Yin, Xin Zhang, Binbin Shi, Tingting Liang, Jian WanAAAI 2025 · DOI

TL;DR

This paper introduces CoT4Rec, a novel framework for recommender systems that leverages Large Language Models (LLMs) and Chain of Thought (CoT) reasoning to generate transparent and accurate user preference profiles. By breaking down the recommendation process into user preference analysis and personalized recommendation stages, CoT4Rec significantly improves recommendation quality and interpretability compared to existing methods.

What Problem Does This Paper Address?

Recommender systems have seen significant advancements with the integration of Large Language Models (LLMs), which excel at understanding and generating text. However, current LLM-based recommender systems face two major challenges: identifier uninterpretability and lack of transparency. Traditional systems often use discrete user and item IDs, failing to leverage the rich semantic information available in LLMs. While some approaches use textual metadata, they often rely on implicit reasoning, which can lead to hallucinations and reduced accuracy. Furthermore, user feedback, often used to infer preferences, can be insufficient or noisy. This paper aims to bridge this gap by developing an LLM-based recommendation architecture that provides a clear recommendation logic and fully utilizes LLM knowledge, while also offering a transparent reasoning path from user behavior to personalized recommendations.

Methods

CoT4Rec employs a two-stage framework: a user preference analyzer and a recommender. The core innovation lies in its CoT Reasoning Strategy for generating user preferences. This strategy involves:

  1. Data Clustering: User historical interaction sequences are segmented into fragments and clustered using Sentence-BERT embeddings. This helps capture representative preference information across different temporal segments.
  2. Demonstration Sampling: For each cluster, a representative sequence is selected, and an LLM (GPT-3.5-turbo-1106) is prompted using a zero-shot approach to generate a user preference summary.
  3. User Preference Construction: These generated preferences are then used as instruction prefixes for LLMs to construct a core logic of preferences for all user fragments. This process aims to create authentic and targeted user preferences.

The preference analyzer is then trained using these LLM-generated preferences as labels. Once trained, it can autonomously generate preference logic without relying on external LLMs, making the system more efficient and flexible. The recommender then utilizes the user's historical interactions, the generated preference logic, and a candidate set to produce personalized recommendations. Both components use the flan-t5-base model as their backbone, fine-tuned independently.

Datasets

The paper evaluates CoT4Rec on four public datasets:

  • MovieLens: ML100K and ML1M, focusing on movie ratings.
  • Amazon: Movies and Electronics categories, utilizing user reviews and ratings.

The paper notes that these datasets were preprocessed, and a leave-one-out strategy was used for evaluation, with the most recent item as the test data and the second most recent as validation data.

Key Insights and Ideas

  • Chain of Thought for User Preferences: The core idea of using CoT to generate user preferences is a significant departure from traditional implicit or explicit preference modeling. It allows for a more nuanced understanding of user behavior by breaking down the reasoning process.
  • Transparency through Preference Analysis: By explicitly generating user preferences as an intermediate step, CoT4Rec offers a more transparent reasoning path compared to black-box models. This makes it easier to understand why a particular recommendation is made.
  • LLMs as Data Augmenters: The framework effectively uses LLMs not just for direct recommendation but as powerful tools to augment user data by generating rich preference profiles.
  • Autonomous Preference Generation: The development of a preference analyzer that can independently generate preference logic after being trained on LLM-generated data is crucial for real-time performance and scalability.
  • Mitigating Hallucinations: The clustering and demonstration sampling approach helps to standardize the reasoning process and reduce the occurrence of hallucinations often associated with zero-shot LLM prompting.

Key Results

CoT4Rec demonstrated superior performance across all four datasets compared to baseline recommender systems. The improvements ranged from 2.2% to 12.2%. Notably, the "Movies" and "Electronics" datasets saw performance gains exceeding 5%. The paper also highlights that CoT4Rec significantly alleviates popularity bias, promoting fairness and diversity in recommendations compared to methods like OpenP5. An ablation study confirmed the effectiveness of both the user preference analyzer and the two-stage recommendation strategy.

Limitations and Future Work

  • Manual Overhead in CoT Construction: While CoT4Rec automates much of the process, the initial construction of CoT prompts and the selection of representative sequences still involve some manual design and potential overhead.
  • Reliance on LLM for Initial Training: The preference analyzer is initially trained using LLM-generated preferences. The quality and efficiency of this initial training phase are critical.
  • Real-time Responsiveness: While the trained preference analyzer addresses real-time issues compared to direct LLM calls, the overall system's latency might still be a consideration for extremely high-throughput applications.

The paper suggests future work could explore more advanced prompt construction approaches and further investigate the interpretability and explainability aspects of the generated recommendations.

Summary updated

BibTeX
@inproceedings{yue2025cot,
  author = {Weiqi Yue and Yuyu Yin and Xin Zhang and Binbin Shi and Tingting Liang and Jian Wan},
  title = {{CoT4Rec: Revealing User Preferences Through Chain of Thought for Recommender Systems}},
  booktitle = {AAAI Conference on Artificial Intelligence},
  year = {2025},
  doi = {10.1609/aaai.v39i12.33434},
}
Figure