Introduction
Standard dialogue models frequently encounter difficulties when dealing with domain-specific logic and out-of-vocabulary (OOV) terms. While Pointer-Generator Networks (PGN) mitigate this by enabling models to copy words from the immediate dialogue context, they remain restricted to the current session's information. In specialized fields like law or customer service, human experts often rely on precedents—recurring patterns from historical data. The Cross Copy Network (CCN) addresses this by enabling "horizontal" retrieval across datasets to guide response generation.
Conceptual Framework: Vertical and Horizontal Copying
The core innovation of CCN lies in its dual-source copying mechanism. "Vertical copy" refers to extracting information from the current conversation's history, a standard practice in PGNs. In contrast, "horizontal copy" allows the model to reference similar historical cases retrieved from the training set. This allows the model to "borrow" complex phrasing, such as legal citations or specific product specifications, that it might otherwise fail to generate correctly.
The Cross Copy Network (CCN) Architecture
The CCN architecture is designed to integrate these two information streams through a multi-stage process:
- Target Case Representation: The current dialogue is encoded using a hierarchical structure consisting of utterance-level Bi-LSTMs, a dialogue-level attention layer, and a Transformer layer to capture immediate context.
- Similar Case Retrieval: Using ElasticSearch, the system identifies the top 50 candidate dialogues from the training data. These are subsequently re-ranked using a fine-tuned RoBERTa model to select the most relevant "Similar Cases" (SC).
- Cross Copy Decoder: The decoder computes two distinct pointer distributions: $\alpha$ for vertical copying (current context) and $\beta$ for horizontal copying (retrieved cases). The final output is a mixture of the standard vocabulary distribution and these two copy mechanisms.
Experimental Evaluation
The model was rigorously tested on two large-scale datasets: the Court Debate Dataset (CDD), containing 121,016 private lending disputes, and the Jing Dong Dialogue Corpus (JDDC), featuring over 1 million e-commerce customer service interactions.
Key Results
CCN achieved state-of-the-art performance across both quantitative and qualitative metrics. On the CDD dataset, the CCN-top3 configuration reached a BLEU score of 24.75, significantly outperforming the standard Transformer (18.58) and PGN (18.75). Human evaluation further confirmed these findings, with CCN outputs receiving higher scores for relevance and fluency. Notably, the model demonstrated a superior ability to replicate complex legal logic by successfully "borrowing" phrasing from historical precedents.
Discussion and Future Work
While CCN represents a significant advancement, error analysis indicates that approximately 50% of failures occur when required information is absent from both the current context and retrieved cases. This suggests that future iterations should incorporate broader external knowledge bases, such as legal textbooks or product manuals. Additionally, addressing the computational overhead of the retrieval step and improving the coherence of long-form responses remain key areas for development.
Conclusion
The Cross Copy Network demonstrates that dialogue generation in specialized domains can be effectively framed as a retrieval-augmented generation problem. By formalizing the distinction between vertical and horizontal copying, CCN provides a robust framework for leveraging historical data to produce more accurate and logically sound dialogue.
Summary updated