TL;DR
This paper presents a practical, three-stage approach to building effective retrieval-based dialogue systems for task-oriented domains like customer service. They introduce a method for semi-automatically creating a large response pool, a novel neural architecture that incorporates dialogue context and business constraints, and a training strategy combining self-supervised learning with human-in-the-loop fine-tuning. This approach leads to a system that is efficient, adaptable, and capable of handling real-world customer interactions.
What Problem Does This Paper Address?
Task-oriented dialogue (TOD) systems are crucial for automating customer service, but deploying them in real-world settings presents significant challenges. Existing end-to-end systems, particularly generative ones, struggle with guaranteeing response quality, style, and privacy. Retrieval-based systems, which select responses from a pre-defined pool, offer more control but need to be highly adaptable to diverse customer needs and business constraints while efficiently handling a vast number of potential responses. The paper addresses the need for a robust, scalable, and adaptable retrieval-based TOD system that can be effectively deployed in industry.
Methods
The authors propose a three-step procedure:
- Semi-Automatic Response Pool Creation: They develop an algorithm to extract a large, representative set of response templates from unannotated historical conversations. This method prioritizes frequency and novelty to ensure good coverage of natural language dialogues without requiring manual annotation.
- Neural Architecture: They enhance the Poly-Encoder architecture. This model uses separate transformers to encode dialogue history, profile features (like order details and business constraints), and candidate responses. Cross-attention layers then learn correlations between these components, and a 2-layer MLP merges history and profile information to represent the complete dialogue context. This architecture allows for efficient caching of response representations.
- Two-Stage Training Strategy:
- Self-Supervised Training (SST): An initial model is trained on a large volume of anonymized human-human conversations. The goal is to rank correct utterances higher than randomly sampled ones, learning from dialogue history and profile features.
- Supervised Fine-Tuning (SFT) with Human-in-the-Loop: The SST model is used in a human-in-the-loop setting. Human agents are presented with top-ranked responses and can accept, modify, or indicate failure. This process generates a smaller, high-quality labeled dataset for fine-tuning the model, ensuring it aligns with business policies and customer needs.
Datasets
- Training Data: Large volumes of anonymized human-human conversations were used for self-supervised training.
- Evaluation Data: The authors used internal conversational datasets from an e-commerce customer service domain for two intents:
- Start-Return (SR): Customers initiating a return.
- Return-Refund Status (RRS): Customers inquiring about existing returns or refunds.
- The datasets were split into training, development, and testing sets.
- Human-in-the-Loop Data: Approximately 67,136 samples were collected over 8 weeks for training and evaluation of the fine-tuned models.
Key Insights and Ideas
- Template Pool Coverage: A large, data-driven template pool, even if semi-automatically generated, can achieve significant conversational coverage, approaching human expert performance.
- Human-in-the-Loop is Crucial: While self-supervised learning provides a strong baseline, human-in-the-loop fine-tuning is essential for adapting the model to specific business constraints and ensuring high-quality, policy-compliant responses.
- Domain Complexity Matters: The performance of dialogue systems varies significantly with domain complexity. More procedural tasks (like starting a return) are easier to automate than open-ended inquiries (like status updates on complex return issues).
- Arbitrary Human Choices: When presented with multiple similar, good options, human agents' choices can appear arbitrary, potentially leading to performance drops if not managed carefully.
- Features are Indispensable: The lack of crucial features (e.g., granular tracking information) can severely limit a model's ability to handle complex scenarios, highlighting the importance of rich contextual information.
- Transfer Learning with PLMs: Leveraging pre-trained language models (like BERT) as encoders significantly improves data efficiency and performance, even with limited supervised data.
Key Results
- Offline Performance: The proposed model achieved strong offline metrics (Recall@k, MRR) on both SR and RRS intents.
- Online Performance:
- The self-supervised model achieved TAR@1 above 70% for SR and around 50% for RRS.
- After supervised fine-tuning, the SR model achieved an Automated Task Completion (ATC) rate of 48.3% in live deployment, approaching human performance in controlled settings.
- Using pre-trained language models (BERT) further boosted performance, with ATC reaching 55.3% for SR.
- Inference Efficiency: The model's architecture allows for linear scaling of inference latency with template set size, enabling real-time conversations even with thousands of templates.
- Template Diversity: An exploration mechanism (Gumbel-Softmax) was shown to help discover less frequent templates, improving diversity without degrading overall performance.
Limitations and Future Work
- RRS Domain: The RRS intent still requires further work for robust deployment, particularly in incorporating essential package tracking information.
- Human Choices: The arbitrary nature of human choices among similar templates needs further investigation to optimize agent productivity and model performance.
- Future Work: The authors plan to deploy the improved BERT model for the RRS intent and further explore its performance and efficiency. They also suggest continued development with pre-trained language models and enhancing the exploration capabilities of the ranking model.
Summary updated