Abstract
DEUX is a novel framework for sociable recommendation dialog systems that tracks conversational attributes, such as movie genres or actors, across both user and system utterances. By predicting which attributes to introduce next and conditionally generating responses, DEUX bridges the gap between rigid task-oriented bots and uncontrollable free-form chatbots. Experimental results demonstrate that DEUX produces highly engaging, consistent, and accurate recommendations compared to existing baselines.
Introduction
Conversational recommendation systems typically struggle to balance two extremes. Task-oriented systems utilize rigid slot-filling (e.g., "What genre do you like?") to narrow down choices. While highly controllable, these systems often result in robotic, monotonous conversations. Conversely, free-form dialog systems generate natural, sociable chit-chat but frequently fail to accurately track user preferences or update dialog states, leading to redundant or nonsensical recommendations.
The core challenge lies in the fact that in natural recommendation dialogue, task-specific content and social content are heavily intertwined. A truly sociable bot must introduce its own opinions and new attributes (e.g., "I love Tom Cruise movies!") to maintain user engagement. Traditional Dialog State Tracking (DST) fails in this context because it typically only tracks user constraints and limits slots to single values. DEUX addresses these limitations by tracking attributes on both sides of the conversation and supporting multiple values per slot.
Methodology
DEUX decouples the dialog system into three distinct modules, allowing for the integration of any off-the-shelf recommender system:
- Attribute Tracking: DEUX extracts entities (genres, actors, titles) from both the user's and the system's dialog history. A BERT-based classifier labels these attributes with positive or negative sentiment (e.g., user likes action, user dislikes horror).
- System Attribute Predictor: This module determines the system's next talking points. Using a GPT-2 model trained on "delexicalized" text (where specific movie names are replaced with generic placeholders), it predicts a new set of attributes for the next turn. An external recommender system then fills these placeholders with actual database items.
- Response Generator: A fine-tuned BlenderBot model generates the final natural language response. This generation is conditional on the attributes predicted in the previous step, ensuring the output contains the correct recommendation data.
Experimental Setup
The primary dataset used for training and evaluation is INSPIRED, which contains 1,011 human-to-human sociable movie recommendation dialogs. To improve the response generator's performance, the authors created an Augmented INSPIRED dataset. By swapping movie attributes with others sharing the same relational structure in the database, they expanded the dataset to 10,000 conversations.
Results and Discussion
DEUX was evaluated against several baselines, including the INSPIRED Bot, a standard BlenderBot, and an ablated version (DEUX-Usr) that only tracked user attributes.
Human Evaluation
In a study involving 58 crowd-workers, DEUX outperformed all baselines across several qualitative metrics. It achieved the highest scores for Consistency (3.64), Naturalness (3.79), Engagingness (3.78), and Sociability (3.86). Furthermore, users engaged in significantly longer conversations with DEUX, averaging 11.91 turns compared to 8.64–10.86 turns for other models.
Recommendation Quality and Language Modeling
DEUX demonstrated superior recommendation accuracy, with only 25.86% of users reporting a poor recommendation fit, compared to 39.66% for the DEUX-Usr ablation. This suggests that system-side tracking is vital for maintaining a consistent persona and avoiding contradictions. Additionally, DEUX achieved a lower (better) perplexity score of 7.42, indicating higher quality and more predictable language generation than its competitors.
Conclusion
DEUX demonstrates that system-side attribute tracking and delexicalized prediction are key to creating sociable, effective recommendation bots. While currently focused on structured attributes like genres and actors, future work will explore expanding the framework to handle unstructured data, such as movie plots, and integrating explicit intent classification to further refine the balance between social and task-oriented content.
Summary updated