DEUX: An Attribute-Guided Framework for Sociable Recommendation Dialog Systems

Yu Li, Shirley Anugrah Hayati, Weiyan Shi, Zhou YuarXiv 2021 · arXiv

Abstract

DEUX is a novel framework for sociable recommendation dialog systems that tracks conversational attributes, such as movie genres or actors, across both user and system utterances. By predicting which attributes to introduce next and conditionally generating responses, DEUX bridges the gap between rigid task-oriented bots and uncontrollable free-form chatbots. Experimental results demonstrate that DEUX produces highly engaging, consistent, and accurate recommendations compared to existing baselines.

Introduction

Conversational recommendation systems typically struggle to balance two extremes. Task-oriented systems utilize rigid slot-filling (e.g., "What genre do you like?") to narrow down choices. While highly controllable, these systems often result in robotic, monotonous conversations. Conversely, free-form dialog systems generate natural, sociable chit-chat but frequently fail to accurately track user preferences or update dialog states, leading to redundant or nonsensical recommendations.

The core challenge lies in the fact that in natural recommendation dialogue, task-specific content and social content are heavily intertwined. A truly sociable bot must introduce its own opinions and new attributes (e.g., "I love Tom Cruise movies!") to maintain user engagement. Traditional Dialog State Tracking (DST) fails in this context because it typically only tracks user constraints and limits slots to single values. DEUX addresses these limitations by tracking attributes on both sides of the conversation and supporting multiple values per slot.

Methodology

DEUX decouples the dialog system into three distinct modules, allowing for the integration of any off-the-shelf recommender system:

  1. Attribute Tracking: DEUX extracts entities (genres, actors, titles) from both the user's and the system's dialog history. A BERT-based classifier labels these attributes with positive or negative sentiment (e.g., user likes action, user dislikes horror).
  2. System Attribute Predictor: This module determines the system's next talking points. Using a GPT-2 model trained on "delexicalized" text (where specific movie names are replaced with generic placeholders), it predicts a new set of attributes for the next turn. An external recommender system then fills these placeholders with actual database items.
  3. Response Generator: A fine-tuned BlenderBot model generates the final natural language response. This generation is conditional on the attributes predicted in the previous step, ensuring the output contains the correct recommendation data.
The DEUX framework architecture, illustrating the flow from dialog context through attribute tracking and prediction to final response generation.

Experimental Setup

The primary dataset used for training and evaluation is INSPIRED, which contains 1,011 human-to-human sociable movie recommendation dialogs. To improve the response generator's performance, the authors created an Augmented INSPIRED dataset. By swapping movie attributes with others sharing the same relational structure in the database, they expanded the dataset to 10,000 conversations.

Results and Discussion

DEUX was evaluated against several baselines, including the INSPIRED Bot, a standard BlenderBot, and an ablated version (DEUX-Usr) that only tracked user attributes.

Human Evaluation

In a study involving 58 crowd-workers, DEUX outperformed all baselines across several qualitative metrics. It achieved the highest scores for Consistency (3.64), Naturalness (3.79), Engagingness (3.78), and Sociability (3.86). Furthermore, users engaged in significantly longer conversations with DEUX, averaging 11.91 turns compared to 8.64–10.86 turns for other models.

Human evaluation results comparing DEUX to baseline models across key metrics including consistency, naturalness, and sociability.

Recommendation Quality and Language Modeling

DEUX demonstrated superior recommendation accuracy, with only 25.86% of users reporting a poor recommendation fit, compared to 39.66% for the DEUX-Usr ablation. This suggests that system-side tracking is vital for maintaining a consistent persona and avoiding contradictions. Additionally, DEUX achieved a lower (better) perplexity score of 7.42, indicating higher quality and more predictable language generation than its competitors.

Conclusion

DEUX demonstrates that system-side attribute tracking and delexicalized prediction are key to creating sociable, effective recommendation bots. While currently focused on structured attributes like genres and actors, future work will explore expanding the framework to handle unstructured data, such as movie plots, and integrating explicit intent classification to further refine the balance between social and task-oriented content.

Summary updated

Abstract

It is important for sociable recommendation dialog systems to perform as both on-task content and social content to engage users and gain their favor. In addition to understand the user preferences and provide a satisfying recommendation, such systems must be able to generate coherent and natural social conversations to the user. Traditional dialog state tracking cannot be applied to such systems because it does not track the attributes in the social content. To address this challenge, we propose DEUX, a novel attribute-guided framework to create better user experiences while accomplishing a movie recommendation task. DEUX has a module that keeps track of the movie attributes (e.g., favorite genres, actors,etc.) in both user utterances and system responses. This allows the system to introduce new movie attributes in its social content. Then, DEUX has multiple values for the same attribute type which suits the recommendation task since a user may like multiple genres, for instance. Experiments suggest that DEUX outperforms all the baselines on being more consistent, fitting the user preferences better, and providing a more engaging chat experience. Our approach can be used for any similar problems of sociable task-oriented dialog system.
BibTeX
@misc{li2021deux,
  author = {Yu Li and Shirley Anugrah Hayati and Weiyan Shi and Zhou Yu},
  title = {{DEUX: An Attribute-Guided Framework for Sociable Recommendation Dialog Systems}},
  year = {2021},
  eprint = {2105.00825},
  archivePrefix = {arXiv},
  url = {https://arxiv.org/abs/2105.00825},
}
Figure