Abstract
Exemplar-Guided Paraphrase Generation (EGPG) aims to rewrite a source sentence to match the syntactic style of an exemplar while preserving original meaning. A major hurdle in this task is the effective disentanglement of content and style. This paper introduces a novel framework utilizing contrastive learning to explicitly supervise the separation of these features. By incorporating Content Contrastive Loss (CCL) and Style Contrastive Loss (SCL), the model achieves state-of-the-art performance on benchmark datasets, significantly reducing content hallucination and improving stylistic fidelity.
Introduction
Paraphrasing with specific stylistic constraints is a complex task for AI. In EGPG, the model receives a source sentence ($X$) and an exemplar ($Z$), producing a target ($Y$) that mirrors $X$'s meaning and $Z$'s syntax. Previous approaches often suffered from "topic drift," where the model inadvertently incorporates content from the exemplar into the output. This failure stems from the inability of standard encoders to perfectly disentangle semantic content from syntactic structure.
Methodology: Contrastive Disentanglement
The proposed architecture addresses disentanglement through a dual-encoder system enhanced by contrastive objectives.
The framework consists of:
- Dual Encoders: A BERT-based encoder ($E_s$) extracts stylistic features, while a GRU-based encoder ($E_c$) captures semantic content.
- Contrastive Losses:
- Content Contrastive Loss (CCL): Ensures the embeddings of the source ($X$) and target ($Y$) are close, while pushing away unrelated sentences.
- Style Contrastive Loss (SCL): Aligns the style embeddings of the exemplar ($Z$) and the target ($Y$), distinguishing them from different syntactic patterns.
- Joint Optimization: The model is trained by minimizing a weighted sum of the standard generation loss (NLL) and the two contrastive losses. This multi-task setup provides stronger supervision signals, mitigating gradient vanishing and forcing the encoders to learn more robust, task-specific features.
Experimental Results
The model was evaluated on the QQP-Pos and ParaNMT datasets. To create training pairs, the authors utilized POS-tag similarity to identify appropriate exemplars.
As shown in the results table, the contrastive approach yields substantial improvements:
- State-of-the-Art Performance: On QQP-Pos, the model achieved a BLEU score of 45.8, a nearly 25% relative improvement over the previous best baseline (SGCP at 36.7).
- Ablation Insights: Removing either CCL or SCL leads to a measurable decline in performance, confirming that both semantic and stylistic supervision are critical.
- Content Fidelity: The authors introduced a Content Matching Accuracy (CMA) metric, demonstrating that their method significantly reduces "topic confusion"—a common issue where models confuse similar entities due to contextual similarities.
Conclusion
By leveraging contrastive learning as a regularizer, this work provides a powerful solution for feature disentanglement in text generation. The resulting model produces paraphrases that are both syntactically accurate to the exemplar and semantically faithful to the source. Future research may extend this framework to more abstract stylistic dimensions, such as sentiment or formality, and explore its application in broader controlled generation tasks.
Summary updated