OpusDistillery: A Configurable End-to-End Pipeline for Systematic Multilingual Distillation of Open NMT Models

Ona de Gibert, Tommi Nieminen, Yves Scherrer, Jörg TiedemannNoDaLiDa/Baltic-HLT 2025 · OpenReview

Abstract

In this work, we introduce OpusDistillery, a novel framework to streamline the Knowledge Distillation (KD) process of multilingual NMT models. OpusDistillery’s main features are the integration of openly available teacher models from OPUS-MT and Hugging Face, comprehensive multilingual support and robust GPU utilization tracking. We describe the tool in detail and discuss the individual contributions of its pipeline components, demonstrating its flexibility for different use cases. OpusDistillery is open-source and released under a permissive license, aiming to facilitate further research and development in the field of multilingual KD for any sequence-to-sequence task.

Introduction

Neural Machine Translation (NMT) has achieved significant quality improvements, yet the most capable models remain computationally expensive and large. This size makes them impractical for edge devices or privacy-focused local translation, such as browser-based translation. Knowledge Distillation (KD)—training a compact "student" model to mimic a large "teacher"—is the standard solution. However, implementing a multilingual distillation pipeline is complex, involving data cleaning, forward translation, alignment extraction, and quantization. OpusDistillery addresses this by providing a standard, configurable tool to automate this entire process.

The OpusDistillery Pipeline

OpusDistillery is built on the Snakemake workflow management system, automating five major stages of the distillation process:

  1. Data Preprocessing: Automates the downloading and cleaning of parallel data, such as the Tatoeba Challenge datasets.
  2. Synthetic Dataset Generation: Leverages teacher models to translate source text. The student learns from these high-quality synthetic translations using techniques like Interpolated Seq-KD.
  3. Student Training: Trains compact architectures (e.g., "tiny" Transformer models) using the Marian toolkit, incorporating guided alignment to improve word-to-word relationship learning.
  4. Exporting: Includes fine-tuning with 8-bit quantization to maximize efficiency for deployment without significant accuracy loss.
  5. Evaluation: Automatically computes standard metrics including BLEU, ChrF, and COMET.
Overview of the OpusDistillery end-to-end pipeline.

A unique feature of the pipeline is its integrated GPU utilization tracking, which allows researchers to monitor energy consumption and the carbon footprint of their training runs.

Experimental Setup

The pipeline was validated using the Tatoeba Translation Challenge for training (sampling up to 10 million sentence pairs per language) and the Flores-200 benchmark for development and testing. The study covered diverse language families, including Finno-Ugric, Romance, and East Slavic.

Results and Discussion

OpusDistillery successfully produced student models with only 16.9 million parameters (approximately 65MB), representing a 12.6x reduction in size compared to standard "Transformer-big" models. While there is a performance trade-off—student models typically score about 5 BLEU points lower than their teachers—the efficiency gains are substantial. Quantized models achieved translation speeds of roughly 3,119 words per second on a single AMD MI250x GPU.

Performance comparison (BLEU) between teacher and student models.

Ablation studies revealed that for many multilingual tasks, a simplified distillation path is highly effective. In some cases, omitting complex steps like Cross-Entropy (CE) filtering or guided alignment resulted in comparable or even superior performance, suggesting that the core distillation process is robust across various configurations (m2o, o2m, and m2m).

Conclusion and Future Work

OpusDistillery provides a robust, open-source framework for compressing multilingual NMT models. Future development will focus on integrating Large Language Models (LLMs) as teachers, incorporating advanced data cleaning tools like BicleanerAI, and implementing word-level distillation strategies to further bridge the performance gap between teacher and student models.

Code Availability: OpusDistillery is available at https://github.com/Helsinki-NLP/OpusDistillery.

Summary updated

Abstract

In this work, we introduce OpusDistillery, a novel framework to streamline the Knowledge Distillation (KD) process of multilingual NMT models. OpusDistillery’s main features are the integration of openly available teacher models from OPUS-MT and Hugging Face, comprehensive multilingual support and robust GPU utilization tracking. We describe the tool in detail and discuss the individual contributions of its pipeline components, demonstrating its flexibility for different use cases. OpusDistillery is open-source and released under a permissive license, aiming to facilitate further research and development in the field of multilingual KD for any sequence-to-sequence task. Our code is available at https://github.com/Helsinki-NLP/OpusDistillery.
BibTeX
@inproceedings{gibert2025opusdistillery,
  author = {Ona de Gibert and Tommi Nieminen and Yves Scherrer and Jörg Tiedemann},
  title = {{OpusDistillery: A Configurable End-to-End Pipeline for Systematic Multilingual Distillation of Open NMT Models}},
  booktitle = {NoDaLiDa/Baltic-HLT},
  year = {2025},
  url = {https://openreview.net/forum?id=m8sMWy2vJm},
}
Figure