Multi-Turn Dialogue Generation in E-Commerce Platform with the Context of Historical Dialogue

Weisheng Zhang, Kaisong Song, Yangyang Kang, Zhongqing Wang, Changlong Sun, Xiaozhong Liu, Shoushan Li, Min Zhang, Luo Si

Abstract

Standard e-commerce chatbots often default to safe but generic responses that fail to address specific customer needs. This paper introduces Conditional Historical Generation (CHG), a model designed to leverage a seller's past conversation history as a dynamic knowledge base. By treating historical dialogues as a searchable resource, the model can "copy" successful past answers—such as specific discount amounts or return policies—to provide more accurate and context-aware customer service. Experimental results on a new e-commerce dataset show that CHG significantly outperforms state-of-the-art baselines in both automatic metrics and human evaluation.

The Generic Response Problem

In e-commerce platforms like Amazon or Taobao, customer service is a critical bottleneck. Automated chatbots are widely used, but they suffer from the "generic response problem." Traditional models, such as the Hierarchical Recurrent Encoder-Decoder (HRED), typically only analyze the current conversation. Consequently, they lack the context of a specific seller’s unique store policies or product-specific knowledge. For instance, while a human seller might provide a specific link for a 5% discount, a generic bot might simply state, "We have many coupons."

Methodology: Conditional Historical Generation (CHG)

The proposed CHG model addresses these limitations through four primary modules designed to integrate historical context into the generation process:

  1. Current Dialogue Encoder: Utilizes a Transformer-based architecture to capture the intent and flow of the ongoing conversation.
  2. Historical Dialogue Encoder: This core innovation employs a two-stage attention mechanism. Customer Attention identifies past customer queries semantically similar to the current one, while Answer Attention selects the most relevant seller responses from those past interactions.
  3. Response Representation Encoder: Encodes candidate responses to ensure stylistic and factual consistency.
  4. Context-Response Attention Decoder: A hybrid decoder featuring a Gated Strategy. It calculates a "gate" value to determine whether to generate a word from a standard vocabulary or "copy" a specific detail (e.g., "6 CNY" or "M size") directly from the relevant historical dialogue.
The CHG model architecture, featuring the Current, Historical, and Response encoders feeding into a gated attention decoder.

Dataset and Experimental Setup

Due to the lack of public datasets containing historical data linked to specific sellers, the authors curated a new dataset from a leading Chinese e-commerce platform. It consists of 60,000 multi-turn service dialogues within the clothing domain. Each dialogue is paired with 1–5 of the seller's most recent historical interactions regarding the same topic or product. On average, dialogues contain 9 utterances, with approximately 27 characters per utterance.

Key Results and Insights

The CHG model demonstrated substantial improvements over state-of-the-art baselines like ReCoSa:

  • Automatic Metrics: CHG achieved a BLEU score of 30.0 (compared to 20.1 for ReCoSa) and a ROUGE-L score of 40.7 (up from 32.9).
  • Diversity: The Distinct-2 metric rose to 0.178, indicating that the model produces more varied and less repetitive language.
  • Human Evaluation: In blind tests, human annotators rated CHG at 2.04 (on a 0-3 scale), noting its ability to provide specific, seller-relevant details.
Qualitative comparison showing how CHG correctly identifies and copies specific refund details (6 CNY) from historical context, whereas other models provide generic or incorrect answers.

The Power of the Copy Mechanism

A key finding is that specific details like refund amounts or shipping times are often "out-of-vocabulary" for standard models. By treating history as a dynamic knowledge base and employing a copy mechanism, the model can provide precise details it wasn't explicitly trained to "know." Furthermore, the authors found that distinguishing between the Customer and Seller roles in historical data is vital to prevent the model from accidentally repeating old questions instead of providing answers.

Limitations and Future Work

While effective, the current study is focused on the clothing domain; adapting it to other sectors like electronics may require refined historical selection strategies. Future research will also explore incorporating customer personalization—such as style preferences or purchase history—to further tailor responses. Additionally, the team aims to improve the model's robustness against "noisy" or outdated historical policies.

Summary updated

Abstract

As an important research topic, customer service dialogue generation tends to generate generic seller responses by leveraging current dialogue information. In this study, we propose a novel and extensible dialogue generation method by leveraging sellers’ historical dialogue information, which can be both accessible and informative. By utilizing innovative historical dialogue representation learning and historical dialogue selection mechanism, the proposed model is capable of detecting most related responses from sellers’ historical dialogues, which can further enhance the current dialogue generation quality. Unlike prior dialogue generation efforts, we treat each seller’s historical dialogues as a list of Customer-Seller utterance pairs and allow the model to measure their different importance, and copy words directly from most relevant pairs. Extensive experimental results show that the proposed approach can generate high-quality responses that cater to specific sellers’ characteristics and exhibit consistent superiority over baselines on a real-world multi-turn customer service dialogue dataset.
BibTeX
@inproceedings{zhang2020multi,
  author = {Weisheng Zhang and Kaisong Song and Yangyang Kang and Zhongqing Wang and Changlong Sun and Xiaozhong Liu and Shoushan Li and Min Zhang and Luo Si},
  title = {{Multi-Turn Dialogue Generation in E-Commerce Platform with the Context of Historical Dialogue}},
  booktitle = {Conference on Empirical Methods in Natural Language Processing},
  year = {2020},
  doi = {10.18653/v1/2020.findings-emnlp.179},
}
Figure