ec: A Reasoning, Reflection, and Refinement Framework for Recommendation Systems

Hao Gu, Rui Zhong, Yu Xia, Wei Yang, Chi Lu, Peng Jiang, Kun GaiRecSys 2025 · OpenReview

TL;DR

This paper introduces R⁴ec, a novel framework that imbues recommendation systems with "System-2" thinking—deliberate, reflective reasoning—by employing an actor-critic architecture. This approach significantly improves recommendation accuracy and revenue by enabling LLMs to iteratively refine their understanding of user preferences and item knowledge, moving beyond basic "System-1" pattern matching.

What Problem Does This Paper Address?

Traditional recommendation systems, even those enhanced by Large Language Models (LLMs), often rely on "System-1" thinking. This means they make quick, intuitive decisions based on learned patterns, making them susceptible to errors in reasoning. A single mistake in inferring user preferences or item knowledge can lead to incorrect recommendations. Furthermore, existing LLM-based methods can be computationally expensive due to numerous API calls. This paper aims to address these limitations by developing a framework that mimics human-like deliberate reasoning, reflection, and refinement to achieve more robust and accurate recommendations.

Methods

The core of R⁴ec is a framework that simulates "System-2" thinking through an iterative process involving two LLM components:

  • Actor Model ($\pi_\theta$): This model is responsible for generating initial reasoning and refining its responses. It's trained on datasets designed to enhance both basic reasoning and refinement capabilities.
  • Reflection Model ($\pi_\psi$): This model acts as a critic, evaluating the actor model's output for reasonableness and providing feedback. It's trained to identify flaws and suggest corrections.

The process works as follows: The actor model generates an initial response (e.g., user preference summary). The reflection model then critiques this response. If flaws are found, the actor model uses this feedback to refine its output. This cycle repeats until the reflection model deems the output reasonable or a maximum number of iterations is reached. This iterative process allows for slow, deliberate reasoning. The framework is applied to both user preference inference and item factual knowledge acquisition. To address computational costs, the authors utilize smaller LLMs (like Qwen-2.5 7B) and employ techniques like LoRA for efficient fine-tuning.

Datasets

The paper constructs new datasets for training the R⁴ec framework:

  • User Preference Reasoning Dataset ($D_{reason}$): Contains user interaction history, target item, and the LLM's reasoned prediction and justification.
  • User Preference Reflection Dataset ($D_{reflect}$): Includes user interaction history, target item, the LLM's reasoned preference, and the reflection model's judgment and feedback on its reasonableness.
  • User Preference Refinement Dataset ($D_{refine}$): Contains the original reasoning, reflection feedback, and the refined user preference and prediction.

Similar datasets are constructed for item factual knowledge. The authors also use standard benchmark datasets for evaluation:

  • Amazon-Book: A subset of the Amazon Review Dataset.
  • MovieLens-1M: A movie recommendation dataset.
  • Industrial Dataset: A large-scale advertising platform dataset.

Key Insights and Ideas

  • System-2 Thinking for Recommendations: The paper convincingly argues that mimicking human-like deliberate reasoning (System-2) through reflection and refinement can significantly outperform simpler, System-1-like LLM approaches in recommendation tasks.
  • Actor-Reflection Architecture: The actor-reflection model pairing is a novel way to introduce self-correction and iterative improvement into LLM-based recommendation systems, moving beyond single-pass generation.
  • Scalability of Reflection: The reflection model's effectiveness doesn't necessarily require it to be as large as the actor model; even a moderately sized reflection model can provide valuable guidance to larger actor models.
  • Iterative Refinement vs. Filtering: While both "Iterative Refinement" and "Reflection as a Filter" strategies improve performance, the iterative approach, which combines reflection with refinement, yields superior results, highlighting the importance of correction.

Key Results

  • Offline Performance: R⁴ec consistently outperforms baseline models and a strong competitor (KAR) across multiple backbone models (DIEN, GRU4Rec, AutoInt, FiGNN, DCN, DeepFM) on both Amazon-Book and MovieLens-1M datasets. For instance, on Amazon-Book, R⁴ec achieved a 2.51% higher relative AUC improvement and a 6.09% greater relative LogLoss reduction compared to the base model when using the Qwen2.5-7B LLM.
  • Online Performance: In a real-world A/B test on an advertising platform, R⁴ec demonstrated a 2.2% increase in revenue and a 1.6% improvement in conversion rate (CVR). For "long-tail" data (users with limited interactions), R⁴ec achieved a 4.1% revenue increase and 3.2% CVR improvement.
  • Model Scaling: The paper shows that increasing the size of the reflection model leads to consistent improvements in AUC and reductions in LogLoss, indicating its crucial role. Similarly, scaling the actor model also benefits from the reflection mechanism.

Limitations and Future Work

  • Computational Cost: While R⁴ec uses smaller LLMs and LoRA, the iterative reflection and refinement process can still be computationally intensive, especially for large-scale industrial applications. The paper mentions tailoring inference strategies for active vs. less active users to manage this.
  • Number of Iterations: The paper notes that while more iterations generally improve performance, the gains diminish, and a single refinement step is adopted as a default due to cost constraints. Future work could explore more efficient ways to determine optimal iteration counts or develop models that converge faster.
  • Generalizability of Reflection: The paper suggests that the reflection model's capacity can become a bottleneck after multiple refinements. Further research could focus on enhancing the reflection model's ability to provide more nuanced and effective feedback.
  • Prompt Engineering: The performance of R⁴ec is dependent on the quality of the prompt templates used for dataset construction. Future work could explore more advanced prompt engineering techniques or automated prompt generation.

Summary updated

BibTeX
@inproceedings{gu2025ec,
  author = {Hao Gu and Rui Zhong and Yu Xia and Wei Yang and Chi Lu and Peng Jiang and Kun Gai},
  title = {{ec: A Reasoning, Reflection, and Refinement Framework for Recommendation Systems}},
  booktitle = {RecSys},
  year = {2025},
  url = {https://openreview.net/forum?id=cKcX25ug7y},
}
Figure