Introduction
Creole languages, emerging from the historical contact between African and European languages during the colonial era, are spoken by millions of people worldwide. Despite their demographic significance, these languages have long suffered from "digital marginalization." Most modern language technologies, including state-of-the-art Machine Translation (MT) systems, provide limited support for Creoles, often focusing only on high-resource examples like Haitian Creole. Kreyòl-MT addresses this gap by introducing the largest cumulative dataset and model suite to date, covering 41 Creole languages across Latin America, the Caribbean, and Africa.
Addressing Digital Marginalization
The lack of representation in NLP research stems from extreme data scarcity and the "domain-locked" nature of existing datasets, which are frequently restricted to religious texts. This narrow focus prevents models from generalizing to everyday speech, news, or legal documentation. Kreyòl-MT seeks to overcome these barriers by curating a massive, multi-genre dataset that supports 172 translation directions.
As shown in the table above, the project includes languages with varying levels of vitality, from institutional languages like Haitian and Nigerian Pidgin to endangered or extinct ones like Garifuna and Berbice Dutch. By providing the first-ever bitexts for 21 of these languages, the researchers have significantly expanded the reach of MT technology.
The Kreyòl-MT Dataset
The core contribution of this work is the Kreyòl-MT dataset, which comprises 14.5 million unique Creole sentences with parallel translations, 11.6 million of which have been publicly released. Unlike previous efforts that relied heavily on the Bible, this dataset incorporates a wide array of genres, including Educational, Legal, Narrative, News, and Wikipedia data.
This genre diversity is the "secret sauce" of the project. By training on a broader spectrum of language use, the resulting models outperform genre-specific systems on general-purpose benchmarks.
Methodology and Modeling
The researchers employed a two-stage pipeline for data acquisition. First, they gathered data by scouring research databases, contacting local experts, and scraping cultural heritage websites. Second, they extracted and aligned sentences from diverse formats, including unsegmented PDFs.
For modeling, the team utilized the YANMTT toolkit to train Transformer models from scratch and fine-tune mBART-50, a pre-trained multilingual model. A key finding during the preprocessing phase was the "cleaning paradox": while aggressive data cleaning benefits high-resource languages, it can be detrimental to "ultra-low-resource" languages where every sentence pair is vital for training signal.
Key Insights and Results
The study highlights the "Creole Family" advantage, where shared morphosyntactic structures across different Creoles facilitate effective cross-lingual transfer. This property allowed the models to demonstrate remarkable zero-shot capabilities; for instance, achieving high performance on Martinican-to-English translation without specific training data for that pair.
Quantitatively, the fine-tuned mBART model established new state-of-the-art results, outperforming previous benchmarks in 26 out of 34 translation directions. On average, the models achieved gains of +6.4 chrF for Creole-to-English and +14.1 chrF for English-to-Creole directions. Even for languages with fewer than 400 parallel sentences, the models produced usable translation scores, proving the efficacy of massive multilingual training for the "long tail" of languages.
Conclusion and Future Work
Kreyòl-MT represents a major step forward in making language technology inclusive for Creole-speaking communities. However, challenges remain, such as managing orthographic evolution and ensuring data comparability across different splits. Future work aims to move beyond text-based systems toward speech-to-speech translation, which would be more accessible to speakers with limited literacy. By proving that genre diversity and cross-lingual transfer can overcome extreme data scarcity, this research provides a blueprint for supporting other underserved language families.
Summary updated