TL;DR
This paper introduces BERTTune, a novel method for fine-tuning Neural Machine Translation (NMT) models using BERTScore as a training objective. By addressing the differentiability challenge of BERTScore, BERTTune achieves statistically significant improvements in translation quality, particularly for metrics that capture semantic similarity.
What Problem Does This Paper Address?
Neural Machine Translation (NMT) models, despite their impressive performance, often suffer from overfitting and limited generalization, especially when training data is scarce. This is partly because they are typically trained to maximize the likelihood of a single reference translation, ignoring valid alternative phrasings. Traditional evaluation metrics like BLEU also have limitations, focusing on exact n-gram matches and missing semantically equivalent translations. While BERTScore offers a more robust evaluation by leveraging contextual embeddings, it's not directly usable as a differentiable training objective. This paper aims to bridge this gap by integrating BERTScore into the NMT training process to improve translation quality and generalization.
Methods
The core innovation of BERTTune lies in making BERTScore, a non-differentiable evaluation metric, usable as a differentiable training objective. The authors propose three methods to generate "soft predictions" from the NMT model, enabling end-to-end differentiability:
- Dense Vectors (DV): This approach computes an "expected embedding" by taking a weighted average of word embeddings from the pretrained language model, where the weights are the probabilities of each word in the vocabulary. This allows the model to consider a broader range of word choices.
- Sparsemax: This method uses the Sparsemax function, which projects the decoder's logits onto the probability simplex, resulting in sparser probability distributions with many zero probabilities. It's fully differentiable.
- Gumbel-Softmax: This technique uses a reparameterization trick with the Gumbel distribution to sample soft categorical variables. A temperature parameter allows controlling the sparsity of the resulting probabilities. This method also offers a degree of exploration through sampling and is fully differentiable.
The fine-tuning process involves minimizing the negative BERTScore (specifically, the FBERT score, which balances precision and recall) using these soft predictions. The NMT model's parameters are updated, while the pretrained language model's parameters are kept fixed.
Datasets
The experiments were conducted on four diverse language pairs:
- German-English (de-en)
- Chinese-English (zh-en)
- English-Turkish (en-tr)
- English-Spanish (en-es)
These datasets were sourced from the IWSLT 2014 shared task. For the de-en dataset, a subset of 7,000 training samples was used for validation, along with specific test sets. For other language pairs, standard validation and test sets were used.
Key Insights and Ideas
- BERTScore as a Training Objective: The paper demonstrates that BERTScore, a metric that correlates well with human judgment, can be effectively used to fine-tune NMT models, leading to improvements beyond traditional n-gram-based metrics.
- Differentiability is Key: The main technical challenge was making BERTScore differentiable. The proposed "soft prediction" methods (DV, Sparsemax, Gumbel-Softmax) are crucial for enabling gradient-based optimization.
- Gumbel-Softmax's Advantage: The Gumbel-Softmax approach, in particular, showed the most consistent improvements, suggesting that its combination of sparsity and sampling might be particularly beneficial for exploring translation variations and improving robustness.
- Sparsity Correlates with Improvement: The authors observed that increased sparsity in the predicted probability vectors (especially with Gumbel-Softmax) generally correlated with better translation accuracy.
- Fine-tuning for a Few Epochs: The experiments indicated that fine-tuning for a limited number of epochs (around 1-2) is optimal to prevent overfitting and achieve the best performance.
- Trade-off and Complementarity: While fine-tuning with BERTScore often improved FBERT and MoverScore, there was sometimes a slight decrease in BLEU score on the validation set. However, on the test set, all BERTTune models outperformed the baseline in BLEU as well, suggesting that the validation and test set distributions might be similar.
Key Results
- BERTTune achieved statistically significant improvements over a strong transformer baseline.
- Improvements of up to 0.58 pp (3.28%) in BLEU score and up to 0.76 pp (0.98%) in BERTScore (FBERT) were observed across diverse language pairs.
- The Gumbel-Softmax (GS) approach yielded the best results, with notable gains for de-en and en-tr.
- Qualitative examples showed BERTTune models better handling nuanced phrases and correctly identifying proper nouns (e.g., "bono" vs. "bonobos").
Limitations and Future Work
- Dataset Size and Baseline Strength: For the en-es dataset, BERTTune did not show significant improvements, possibly because the baseline model was already very strong, making it harder to improve.
- Exploration of Factors: The authors plan to explore the impact of factors like dataset size, the degree of sparsity in predictions, and the choice of different pretrained language models.
- Advanced Training Strategies: Future work will also investigate using beam search or sequential sampling during training to further enhance exploration of candidate translations.
Summary updated