Abstract
Standard e-commerce chatbots often default to safe but generic responses that fail to address specific customer needs. This paper introduces Conditional Historical Generation (CHG), a model designed to leverage a seller's past conversation history as a dynamic knowledge base. By treating historical dialogues as a searchable resource, the model can "copy" successful past answers—such as specific discount amounts or return policies—to provide more accurate and context-aware customer service. Experimental results on a new e-commerce dataset show that CHG significantly outperforms state-of-the-art baselines in both automatic metrics and human evaluation.
The Generic Response Problem
In e-commerce platforms like Amazon or Taobao, customer service is a critical bottleneck. Automated chatbots are widely used, but they suffer from the "generic response problem." Traditional models, such as the Hierarchical Recurrent Encoder-Decoder (HRED), typically only analyze the current conversation. Consequently, they lack the context of a specific seller’s unique store policies or product-specific knowledge. For instance, while a human seller might provide a specific link for a 5% discount, a generic bot might simply state, "We have many coupons."
Methodology: Conditional Historical Generation (CHG)
The proposed CHG model addresses these limitations through four primary modules designed to integrate historical context into the generation process:
- Current Dialogue Encoder: Utilizes a Transformer-based architecture to capture the intent and flow of the ongoing conversation.
- Historical Dialogue Encoder: This core innovation employs a two-stage attention mechanism. Customer Attention identifies past customer queries semantically similar to the current one, while Answer Attention selects the most relevant seller responses from those past interactions.
- Response Representation Encoder: Encodes candidate responses to ensure stylistic and factual consistency.
- Context-Response Attention Decoder: A hybrid decoder featuring a Gated Strategy. It calculates a "gate" value to determine whether to generate a word from a standard vocabulary or "copy" a specific detail (e.g., "6 CNY" or "M size") directly from the relevant historical dialogue.
Dataset and Experimental Setup
Due to the lack of public datasets containing historical data linked to specific sellers, the authors curated a new dataset from a leading Chinese e-commerce platform. It consists of 60,000 multi-turn service dialogues within the clothing domain. Each dialogue is paired with 1–5 of the seller's most recent historical interactions regarding the same topic or product. On average, dialogues contain 9 utterances, with approximately 27 characters per utterance.
Key Results and Insights
The CHG model demonstrated substantial improvements over state-of-the-art baselines like ReCoSa:
- Automatic Metrics: CHG achieved a BLEU score of 30.0 (compared to 20.1 for ReCoSa) and a ROUGE-L score of 40.7 (up from 32.9).
- Diversity: The Distinct-2 metric rose to 0.178, indicating that the model produces more varied and less repetitive language.
- Human Evaluation: In blind tests, human annotators rated CHG at 2.04 (on a 0-3 scale), noting its ability to provide specific, seller-relevant details.
The Power of the Copy Mechanism
A key finding is that specific details like refund amounts or shipping times are often "out-of-vocabulary" for standard models. By treating history as a dynamic knowledge base and employing a copy mechanism, the model can provide precise details it wasn't explicitly trained to "know." Furthermore, the authors found that distinguishing between the Customer and Seller roles in historical data is vital to prevent the model from accidentally repeating old questions instead of providing answers.
Limitations and Future Work
While effective, the current study is focused on the clothing domain; adapting it to other sectors like electronics may require refined historical selection strategies. Future research will also explore incorporating customer personalization—such as style preferences or purchase history—to further tailor responses. Additionally, the team aims to improve the model's robustness against "noisy" or outdated historical policies.
Summary updated