Tradutor: Building a Variety Specific Translation Model

Hugo O. Sousa, Satya Almasian, Ricardo Campos, Alípio Mário JorgeAAAI 2025 · DOI

Abstract

European Portuguese (PT-EU) is frequently underrepresented in Large Language Models (LLMs) due to the overwhelming prevalence of Brazilian Portuguese (PT-BR) data. This paper presents Tradutor, a novel methodology and the largest open-source dataset (PTradutor) specifically designed to address this "majority bias." By utilizing a "retro-translation" technique to convert monolingual PT-EU data into a synthetic parallel corpus, the researchers fine-tuned state-of-the-art LLMs like LLaMA-3. The resulting models significantly outperform existing open-source systems and approach the performance of industry-leading proprietary tools in variety-specific translation accuracy.

Introduction

The dominance of Brazilian Portuguese in training corpora—accounting for over 70% of native speakers—leads to a significant linguistic bias in standard translation tools. These systems often default to PT-BR grammar and vocabulary, even when the intended target is European Portuguese. This lack of precision is particularly problematic in specialized fields such as healthcare and the judiciary in Portugal and Mozambique. The primary challenge in rectifying this has been the scarcity of high-quality, large-scale parallel datasets (English to PT-EU). Tradutor overcomes this by generating a massive synthetic dataset and applying targeted instruction fine-tuning.

Methods

The researchers developed a three-step pipeline to bridge the data gap:

  1. Retro-translation: To circumvent the lack of parallel data, the team collected a vast amount of monolingual European Portuguese text. They then used a high-quality, variety-agnostic translation engine (Google Translate) to translate this text into English. This creates a synthetic parallel corpus where the target (PT-EU) is authentic human text, and the source (English) is machine-generated.
  2. Rigorous Filtering: To ensure data quality and variety purity, an extensive filtering pipeline was applied. This included boilerplate removal using the jusText library, ASCII character checks to confirm the presence of Portuguese-specific diacritics, and length filtering to optimize the training samples.
  3. Instruction Fine-Tuning: The resulting PTradutor dataset was used to fine-tune three state-of-the-art LLMs: Gemma-2 (2B), Phi-3-mini (3.8B), and LLaMA-3 (8B). The study compared Full Fine-Tuning (FFT) against Parameter-Efficient Fine-Tuning (PEFT/LoRA) to evaluate their impact on translation quality and variety adherence.

Datasets

The core contribution of this work is the PTradutor training dataset. It contains 1,719,002 documents spanning multiple domains, making it the largest publicly available English-PT-EU parallel corpus.

Breakdown of the PTradutor dataset by domain and token counts.

For evaluation, the researchers utilized the FRMT and NTrex benchmarks. These datasets are specifically curated with human translations labeled for regional varieties, allowing for a precise assessment of the models' ability to distinguish between PT-EU and PT-BR.

Key Insights and Metrics

A critical insight of the study was the "Variety-Blind Bridge." The authors observed that while translation engines often struggle to produce a specific variety, they are remarkably consistent when translating from different varieties into English. PT-EU and PT-BR texts resulted in identical English translations approximately 87% of the time, validating the retro-translation approach.

The study also highlighted the trade-offs between fine-tuning methods. While LoRA-trained models achieved high Variety Identification (VID) scores—a metric developed to classify if output "sounds" like PT-EU—they were prone to "repetition loops." In contrast, Full Fine-Tuning (FFT) yielded much higher coherence and overall translation quality. Furthermore, the authors noted a potential bias in learnable metrics like COMET, which may undervalue PT-EU improvements due to being trained on general (predominantly PT-BR) Portuguese data.

Results

The fine-tuned LLaMA-3 (8B) model emerged as the superior system, outperforming all other open-source baselines across all metrics.

Performance comparison of various models on FRMT and NTrex benchmarks.

On the FRMT benchmark, LLaMA-3 (FFT) achieved a BLEU score of 41.12, a significant improvement over its zero-shot performance (31.47). The model's performance approached that of industry-standard closed-source models like DeepL. Most importantly, the dramatic increase in VID scores across all fine-tuned models proves that the methodology successfully taught the specific nuances of European Portuguese grammar and lexicon.

Conclusion and Future Work

Tradutor demonstrates a scalable path for improving translation quality for underrepresented language varieties. While the study focused on models up to 8B parameters for accessibility, the authors suggest that applying these methods to larger models (e.g., 70B+) could further close the gap with proprietary systems. Future research will also explore advanced decoding strategies like beam search and prompt optimization to further enhance fluency. The PTradutor dataset and model checkpoints have been released to the community to encourage further development in variety-specific NLP.

Summary updated

BibTeX
@inproceedings{sousa2025tradutor,
  author = {Hugo O. Sousa and Satya Almasian and Ricardo Campos and Alípio Mário Jorge},
  title = {{Tradutor: Building a Variety Specific Translation Model}},
  booktitle = {AAAI Conference on Artificial Intelligence},
  year = {2025},
  doi = {10.1609/aaai.v39i24.34704},
}
Figure