Contrastive Preference Learning for Neural Machine Translation
Lists
Only you see this
Abstract
There exists a discrepancy between the token-level objective during training and the overall sequence-level quality that is expected from the model. This discrepancy leads to issues like exposure bias.To align the model with human expectations, sequence-level objectives are often used to fine-tune pre-trained models.In this paper, we introduce a contrastive preference model that enhances the traditional Plackett-Luce model by incorporating an indicator function. Building upon this novel preference model, we propose Contrastive Preference Learning (CPL), which uses offline samples with list-wise preferences to fine-tune a pre-trained model in Neural Machine Translation. Our experiments, conducted on three language pairs, demonstrate that CPL outperforms not only the vanilla Transformer model but also other token-level and sequence-level baselines. Furthermore, the ablation study highlights the essential role of the proposed indicator function in achieving this improvement.
BibTeX
@inproceedings{he2024contrastive,
author = {Jianfei He and Shichao Sun and Sen Peng and Jie Xu and Xiaohua Jia and Wenjie Li},
title = {{Contrastive Preference Learning for Neural Machine Translation}},
booktitle = {Conference of the North American Chapter of the Association for Computational Linguistics},
year = {2024},
doi = {10.18653/v1/2024.findings-naacl.174},
}