Conversational Recommendation System with Unsupervised Learning

Yueming Sun, Yi Zhang, Yunfei Chen, Roger Jin

Abstract

Traditional conversational agents require extensive hand-labeled datasets to understand user intent, creating a significant barrier to entry for new business domains. This paper presents a conversational recommendation system that bypasses this "labeling bottleneck" by using unsupervised deep learning. By training on unlabeled chat logs and utilizing successful transactions as "delayed rewards," the system learns to track dialogue states and recommend products effectively. The proposed modular architecture integrates a personalized recommendation engine directly into the dialogue policy, enabling natural, goal-oriented interactions that adapt to new domains with minimal human intervention.

The Challenge: The Labeling Bottleneck

Developing a virtual sales assistant is notoriously difficult because standard dialogue systems rely on tens of thousands of manually annotated examples or rigid, hand-crafted rules. While recommendation systems can easily adapt to new domains using transaction data or clickstreams, dialogue systems are often domain-specific and expensive to retrain. This research aims to bridge that gap by creating an agent that learns how to interact and recommend by observing human-to-human conversations and their eventual outcomes. The goal is a system that can be launched in a new business category without the massive overhead of manual labeling.

System Architecture

The authors propose a modular framework that integrates recommendation logic into the core dialogue policy. The system operates through four primary components that work in tandem to process user input and generate appropriate responses:

  1. Language Understanding: This initial stage translates raw user text into an abstract representation that the system can process.
  2. User Intention Prediction and State Tracking: This core innovation uses unsupervised deep learning to identify user goals. Instead of relying on predefined "slot-value" labels (e.g., cuisine = Italian), the model learns to identify relevant features by analyzing successful transaction outcomes.
  3. Policy & Recommendation Engine: This component determines the agent's next move. Unlike traditional bots where recommendation is a separate search step, here the recommendation engine is a core part of the policy. It ranks products based on user history, demographics, and context gathered during the chat.
  4. Natural Language Generation (NLG): The system employs a hybrid approach. It uses template-based models for high-precision tasks (like order confirmation) and neural sequence-to-sequence models to maintain a more natural, fluid conversational flow.
The modular architecture of the conversational recommendation system, showing the flow from user input to system response.

Learning from Delayed Rewards

The system's primary innovation is its ability to learn from the "destination" of a conversation. By treating a completed purchase as a training signal—a "delayed reward"—the model can back-reference the preceding unlabeled dialogue.

For instance, if a user successfully completes an order for a "hot latte," the system can infer that the earlier, unlabeled parts of the chat were successfully identifying the necessary parameters (e.g., product type, temperature, and size). This allows the system to map natural language to specific intents and "slots" without requiring a human to label every sentence in the training set. This approach effectively turns transaction data into a supervision signal for the dialogue model.

Implementation and Results

The system was implemented within the Rul.ai platform to demonstrate its real-world feasibility. A key use case highlighted in the paper involves a multi-step coffee ordering process. In this demo, the agent successfully manages various user preferences—such as size, temperature, and quantity—and collects delivery details within a fluid chat interface.

A demonstration of the conversational agent handling a multi-step coffee order, including preference gathering and final order confirmation.

The results indicate that the system can adapt to new domains, such as transitioning from restaurant discovery to coffee ordering, without requiring manual rule-writing. By treating the recommendation engine as a core part of the dialogue policy, the system ensures that every interaction is fundamentally driven by the ultimate goal of fulfilling a user's needs.

Limitations and Future Outlook

While the unsupervised approach is promising, it relies heavily on a high volume of successful transactions to provide sufficient training signals. Future work may explore how to incorporate "negative" data from failed conversations to further refine the policy and help the agent avoid common pitfalls. Additionally, while the system excels in structured tasks like coffee ordering, more complex or ambiguous domains may require further research into mapping abstract intents to transaction data. The authors suggest that the hybrid NLG approach provides a pragmatic path for production systems, balancing the safety of templates with the naturalness of neural models.

Summary updated

BibTeX
@inproceedings{sun2016conversational,
  author = {Yueming Sun and Yi Zhang and Yunfei Chen and Roger Jin},
  title = {{Conversational Recommendation System with Unsupervised Learning}},
  booktitle = {RecSys},
  year = {2016},
  doi = {10.1145/2959100.2959114},
}
Figure