Abstract
Exemplar-Guided Paraphrase Generation (EGPG) is the task of generating a paraphrase that retains the semantic meaning of a source sentence while adopting the syntactic structure of a provided exemplar. The primary challenge lies in disentangling content from style within latent representations. We propose a novel framework that integrates contrastive learning into an encoder-decoder architecture to enforce this separation. By applying specific content and style contrastive losses, our model forces the encoders to specialize in their respective domains. Our approach demonstrates significant improvements over state-of-the-art baselines on the QQP-Pos and ParaNMT benchmarks, producing paraphrases that are more semantically accurate and stylistically consistent.
Introduction
Paraphrasing with stylistic control requires a model to merge the "what" (content) of a source sentence with the "how" (style) of an exemplar. Traditional neural models often suffer from "content drift" or "style mismatch" because they fail to properly isolate these two components. When style and content are entangled in the latent space, the decoder may inadvertently alter the meaning or ignore the target structure. This paper addresses this bottleneck by leveraging contrastive representation learning to achieve robust disentanglement.
Methodology
The proposed framework utilizes a dual-encoder setup to process the source and exemplar sentences independently.
Architecture
As illustrated in Figure 1, the model consists of a content encoder ($E_c$), a style encoder ($E_s$), and a shared decoder ($D$). The content encoder is tasked with extracting semantic features, while the style encoder captures syntactic patterns.
Contrastive Objectives
To ensure the encoders specialize, we introduce two auxiliary losses:
- Content Contrastive Loss (CCL): This objective ensures that the content embeddings of the source sentence ($X$) and its target paraphrase ($Y$) are similar, as they share the same meaning. It penalizes the model if these embeddings diverge or if they become too similar to unrelated sentences in the training batch.
- Style Contrastive Loss (SCL): Similarly, the SCL aligns the style embeddings of the target sentence ($Y$) with the exemplar ($Z$). This forces the model to replicate the syntactic template of the exemplar in the generated output.
The final training objective is a joint optimization of the standard generation loss (negative log-likelihood) and these two contrastive terms. This multi-task approach also helps mitigate gradient vanishing by providing more direct supervision to the encoders.
Experimental Results
We evaluated our model on two benchmark datasets: QQP-Pos (derived from Quora Question Pairs) and ParaNMT (generated via back-translation).
Quantitative Performance
Our model significantly outperformed existing baselines such as SCPN, SGCP, and CGEN across all automatic metrics. As shown in Figure 6, the improvements are particularly pronounced on the QQP-Pos dataset, where our model achieved a BLEU score of 45.8, compared to the previous best of 36.7.
Ablation Studies and Human Evaluation
Ablation studies confirmed that both CCL and SCL contribute to the model's success. Removing either component led to a drop in performance, particularly in Content Matching Accuracy (CMA). Furthermore, human evaluations rated our model's outputs higher for both fluency and stylistic adherence, indicating that the quantitative gains reflect genuine improvements in text quality.
Conclusion
By incorporating contrastive learning, we have developed a more effective method for disentangling content and style in exemplar-guided paraphrase generation. Our model not only achieves higher accuracy on standard benchmarks but also demonstrates better structural adherence to stylistic templates. Future work will focus on improving data quality for diverse training and extending this contrastive framework to other controllable generation tasks like sentiment-based style transfer.
Summary updated