Introduction
Modern dialogue systems, particularly those based on Transformers or Pointer-Generator Networks, have achieved high levels of fluency. However, they often struggle with "dialogue logic"—the domain-specific structural patterns essential in professional settings like legal proceedings or customer service. In these environments, conversations often follow predictable logical flows that standard models, treating each dialogue as an isolated event, fail to capture.
To address this, the Cross Copy Network (CCN) introduces a dual-pointer mechanism. Unlike traditional models that only reference the current conversation history, CCN "looks over the shoulder" at similar historical cases to inform its generation. By implementing both vertical copy (from the current context) and horizontal copy (from similar external dialogues), the model captures latent knowledge and professional patterns without the need for expensive, manually-curated knowledge graphs.
Methodology
The CCN architecture is built upon a hierarchical encoder and a multi-stage decoder designed to integrate internal and external information seamlessly.
Hierarchical Encoding
The model processes the "Target Case" (the current dialogue) through three distinct layers:
- Utterance Layer: Captures word-level information while explicitly embedding speaker roles (e.g., Judge vs. Plaintiff), acknowledging that different roles have distinct lexical spaces.
- Dialogue Layer: Captures dependencies and flow between consecutive sentences.
- Transformer Layer: Models long-range dependencies across the entire conversation.
Similar Case Retrieval
To enable horizontal copying, the system employs a retrieval-and-rank pipeline. It uses ElasticSearch to find candidate dialogues from a large corpus and then utilizes a fine-tuned RoBERTa model to identify the most relevant "Similar Cases" (SCs).
The Cross Copy Mechanism
The decoder calculates two distinct pointer distributions to determine the next word:
- Vertical Copy ($\alpha$): Determines the probability of copying a word from the current dialogue's history, such as specific names, dates, or monetary amounts.
- Horizontal Copy ($\beta$): Determines the probability of copying phrases or logical structures from the retrieved similar cases, such as standard legal phrasing or inquiry logic.
The final output is a mixture of these two distributions and a standard vocabulary distribution, allowing the model to generate out-of-vocabulary words if they appear in either the current context or the retrieved cases.
Experimental Setup
The authors evaluated CCN on two large-scale, domain-specific datasets:
- Court Debate Dataset (CDD): Comprising 121,016 records of private lending disputes. The task is to generate the judge’s next utterance based on the interaction between the plaintiff and defendant.
- Jing Dong Dialogue Corpus (JDDC): An e-commerce dataset with over 1 million multi-turn dialogues, focused on generating customer service responses.
Results and Discussion
CCN consistently outperformed state-of-the-art baselines, including standard Transformers and Pointer-Generator Networks (PGN), across both datasets.
Quantitative Performance
On the CDD (Legal) dataset, the CCN (top-3 similar cases) achieved a BLEU score of 24.75, a significant improvement over PGN (18.75) and standard Transformers (18.58). Similar gains were observed on the JDDC dataset, where CCN reached 19.53 BLEU, compared to PGN's 14.37. The results indicate that performance scales positively with the number of similar cases referenced, though this increases computational overhead.
Qualitative Analysis
Human evaluators rated CCN higher in both relevance and fluency. Case studies revealed that CCN was uniquely capable of replicating complex legal inquiry logic by successfully "horizontal copying" from historical precedents. However, the model still faces a "knowledge gap"—approximately 53% of errors in the legal dataset occurred when the necessary information (such as a specific article of law) was absent from both the current context and the retrieved cases.
Conclusion
The Cross Copy Network demonstrates that retrieval-based "soft knowledge" is a powerful alternative to rigid knowledge graphs for specialized dialogue generation. By leveraging the repetitive nature of professional logic through horizontal copying, CCN bridges the gap between fluency and domain-specific accuracy. Future research will explore multi-granularity copying to move beyond word-level pointers toward entire semantic templates.
Summary updated