TL;DR
This paper introduces CRAVE, a novel framework that enhances Large Language Model (LLM)-based conversational recommender systems (CRS) by leveraging "collaborative verbalized experience." CRAVE trains LLM agents to learn from historical conversations and user feedback, enabling them to generate more personalized and diverse recommendations, particularly in open-ended debate scenarios.
What Problem Does This Paper Address?
Traditional recommender systems often struggle to capture nuanced user preferences, especially in conversational settings where users express themselves naturally. While Large Language Models (LLMs) have shown impressive zero-shot capabilities in conversational recommendation, effectively utilizing historical conversations and user feedback remains a challenge. Existing methods either rely on simple few-shot examples or global rules, which fail to capture implicit, preference-oriented knowledge. Furthermore, LLM-based agents can sometimes exhibit convergent thinking, leading to less diverse recommendations. This paper aims to bridge this gap by developing a system that learns from past interactions to inform future recommendations, fostering more divergent and personalized suggestions.
Methods
CRAVE introduces a multi-agent system with three core components:
- Conversational Recommendation Agents (CRAs): These are LLM-based agents designed to generate recommendations. The paper explores two configurations:
- Chain-of-Thought (COT) Agent: A single LLM agent that reasons through the conversation and provides recommendations.
- Debater-Critic Agent (DCA) System: A more sophisticated system where multiple "debater" agents engage in a dialogue to refine reasoning and recommendations, with a "critic" agent evaluating their output. This is designed to encourage divergent thinking.
- Collaborative Experience Retrieval: This module is central to CRAVE's innovation. It involves:
- Trajectory Sampling: LLM agents interact with historical conversations, and their actions (reasoning and recommendations) are recorded.
- Verbalized Experience Collection: The system reflects on these trajectories, using user feedback (ground truth items) to summarize useful experiences in natural language. These experiences are stored in agent-specific "experience banks."
- Collaborative Retrieval Network: A fine-tuned Sentence-BERT model is used to retrieve relevant verbalized experiences from the banks based on the similarity of user preferences in new conversations. This network is further fine-tuned using item content and a novel collaborative similarity metric.
- Experience Augmented Generation: The retrieved verbalized experiences are then used to augment the LLM agents' prompts, guiding their reasoning and recommendation generation process.
Datasets
The paper evaluates CRAVE on two widely used real-world conversational recommendation datasets:
- Redial: A dataset for conversational recommendation.
- Reddit-v2: A movie-name corrected version of the Reddit dataset, also used for conversational recommendation.
The original test sets were split into validation and test sets to tune the collaborative retrieval model and evaluate the final system, respectively.
Key Insights and Ideas
- Verbalized Experience is Key: Simply retrieving few-shot examples isn't enough. The system needs to "learn" from past interactions by reflecting on agent actions and user feedback to build a rich understanding of user preferences.
- Debate Fosters Diversity: The Debater-Critic Agent (DCA) system, by design, encourages divergent thinking, leading to more comprehensive coverage of user preferences compared to a single COT agent, especially when augmented with collaborative experience.
- Collaborative Similarity Matters: A novel metric that considers both conversational similarity and item content is crucial for effectively retrieving relevant past experiences. This goes beyond simple semantic similarity.
- LLMs Benefit from Structured Learning: Even powerful zero-shot LLMs can be significantly improved by providing them with curated, experience-based guidance, particularly in complex, preference-driven tasks like recommendation.
Key Results
- CRAVE, with both COT and DCA configurations, consistently outperforms state-of-the-art baselines, including zero-shot LLMs, on both the Reddit-v2 and Redial datasets.
- The DCA system, when augmented with CRAVE's collaborative experience, shows better coverage of ground truth items compared to the COT agent.
- Ablation studies confirm the effectiveness of both the verbalized experience collection module and the collaborative retrieval network. Removing or simplifying these components leads to performance degradation.
- Qualitative analysis shows that CRAVE's retrieved experiences help LLMs better align with user preferences, broaden thematic scope, and clarify user intent, leading to more relevant and diverse recommendations.
Limitations and Future Work
- LLM Dependency: CRAVE's effectiveness relies on the availability of large LLMs with emergent reasoning abilities. It may not be directly applicable to smaller PLMs that require fine-tuning.
- Computational Overhead: Augmenting LLM inputs with retrieved experiences can increase inference latency.
- Bias Inheritance: LLMs and the retrieval network can inherit biases from their training data. The authors suggest filtering training conversations as a mitigation strategy.
Future work could explore applying CRAVE to other downstream tasks that involve conversational agents and investigate more efficient methods for experience retrieval and augmentation to reduce latency.
Summary updated