Efficient Machine Translation Domain Adaptation

Pedro Martins, Zita Marinho, André F. T. Martins2022 · DOI

Abstract

Machine translation models struggle when translating out-of-domain text, which makes domain adaptation a topic of critical importance. However, most domain adaptation methods focus on fine-tuning or training the entire or part of the model on every new domain, which can be costly. On the other hand, semi-parametric models have been shown to successfully perform domain adaptation by retrieving examples from an in-domain datastore. A drawback of these retrieval-augmented models, however, is that they tend to be substantially slower. In this paper, we explore several approaches to speed up nearest neighbor machine translation. Translation quality and runtimes for several domains show the effectiveness of the proposed solutions.

Introduction

Standard Neural Machine Translation (NMT) models are parametric, meaning their knowledge is fixed within model weights during training. Consequently, they often struggle with out-of-domain data, such as medical texts when the model was primarily trained on news corpora. A popular solution is $k$NN-MT, a semi-parametric approach that augments the base model with a "datastore" of domain-specific translation examples. During generation, the model retrieves the $k$-nearest neighbors from this datastore to refine its predictions.

While $k$NN-MT significantly improves accuracy, the retrieval phase creates a massive computational bottleneck, often making the system 100x slower than standard models. This paper introduces Efficient $k$NN-MT, a suite of optimizations designed to slash search time while maintaining the quality gains of retrieval-augmented translation.

Methodology

The authors propose three primary strategies to optimize the $k$NN-MT pipeline:

  1. Datastore Pruning: Using "greedy merging," the system identifies entries with similar mathematical representations (keys) for the same word and merges them. This reduces the total size of the datastore without losing critical information.
  2. Dimension Reduction (PCA): High-dimensional word vectors (typically 1024D) are compressed to 256D using Principal Component Analysis (PCA). Smaller vectors allow for significantly faster distance calculations during the search process.
  3. Retrieval Distributions Cache: This novel contribution leverages the observation that translation contexts are often repetitive, especially during beam search. By caching recent lookup results and reusing them when the current context is sufficiently similar (measured by Euclidean distance), the model can skip the expensive full datastore search.
Ablation study showing the cumulative speed benefits of PCA, pruning, and the retrieval cache compared to vanilla kNN-MT.

Experimental Setup

The methods were evaluated across four distinct domains: Medical, Law, IT, and Koran. The researchers used a pre-trained WMT’19 German-English model as the base, with domain-specific datastores ranging from approximately 18,000 to 467,000 sentences.

Key Results and Insights

Speed and Scalability

Efficient $k$NN-MT is approximately twice as fast as vanilla $k$NN-MT. A major advantage of this approach is its scalability. Unlike previous "Fast $k$NN" implementations that often crash or face severe memory limits at batch sizes larger than 2, the proposed Efficient $k$NN-MT scales effectively to batch sizes of 16 and beyond, greatly increasing total system throughput.

Generation speed across four domains as a function of batch size, comparing the base model, vanilla kNN-MT, and the proposed Efficient kNN-MT.

Quality and Efficiency Trade-offs

Despite the aggressive optimizations, the model maintains high translation quality. It outperforms the base MT model by over 8 BLEU points and 5 COMET points. The retrieval cache proved particularly effective, allowing the system to skip the full datastore search 57% of the time on average.

Conclusion

Efficient $k$NN-MT demonstrates that the computational overhead of semi-parametric translation can be significantly mitigated through intelligent pruning, compression, and caching. While the authors found that simple "adaptive retrieval" (predicting when to skip retrieval entirely) was prone to error propagation, the current caching strategy provides a robust middle ground. Future work may explore more sophisticated methods to determine when the base model's confidence is sufficient to bypass the retrieval step without sacrificing accuracy.

Summary updated

Abstract

Machine translation models struggle when translating out-of-domain text, which makes domain adaptation a topic of critical importance. However, most domain adaptation methods focus on fine-tuning or training the entire or part of the model on every new domain, which can be costly. On the other hand, semi-parametric models have been shown to successfully perform domain adaptation by retrieving examples from an in-domain datastore A drawback of these retrievalaugmented models, however, is that they tend to be substantially slower. In this paper, we explore several approaches to speed up nearest neighbor machine translation. We adapt the methods recently proposed by He et al. ( Translation quality and runtimes for several domains show the effectiveness of the proposed solutions. 1
BibTeX
@misc{martins2022efficient,
  author = {Pedro Martins and Zita Marinho and André F. T. Martins},
  title = {{Efficient Machine Translation Domain Adaptation}},
  year = {2022},
  doi = {10.18653/v1/2022.spanlp-1.3},
}
Figure