Abstract
Emakhuwa is the most widely spoken language in Mozambique, yet it remains a low-resource language in the digital sphere. This paper presents the first comprehensive suite of NLP resources for Emakhuwa, including manually translated news corpora, OCR-derived datasets, and benchmarks for Machine Translation (MT) and News Classification. Our results demonstrate that modest amounts of high-quality and synthetic data can significantly bridge the performance gap for low-resource Bantu languages.
Introduction
Despite being spoken by approximately 7.5 million people—roughly 25% of Mozambique's population—Emakhuwa is digitally marginalized. It lacks the massive datasets required to train modern AI models for tasks like Machine Translation or News Classification. The challenges are both linguistic and structural: Emakhuwa is an agglutinative Bantu language with complex tonal attributes and no standardized orthography. Furthermore, existing data is often trapped in physical archives or scattered across small-scale digital platforms. This work aims to provide the foundational datasets and benchmarks needed to build useful tools for Emakhuwa speakers.
Methods
We employed a multi-pronged approach to resource building and benchmarking:
- Human-in-the-loop Translation: Professional translators were hired to create a high-quality parallel corpus between Portuguese and Emakhuwa.
- Data Augmentation: To expand the training set, we utilized Optical Character Recognition (OCR) on a bilingual book (Método Macua) and back-translation. The latter involved using an initial MT model to translate monolingual Emakhuwa text back into Portuguese to create synthetic training pairs.
- Model Benchmarking:
- For Machine Translation, we compared a standard Transformer baseline against fine-tuned versions of massive multilingual models like NLLB-200, M2M-100, and byT5. We specifically included African-centric models like AfribyT5 to evaluate the impact of regional pre-training.
- For News Classification, we tested both classical Machine Learning (Naive Bayes, XGBoost) and modern Language Models (AfriBERTa, XLM-R).
Datasets
We introduce several new resources, released under a CC BY 4.0 license, to support the Emakhuwa NLP ecosystem. These include a News Bitext of 18,540 parallel sentences across seven categories, OCR-extracted data from printed books, and monolingual data sourced from Wòniherya News and the Wikimedia Incubator.
As shown in the table above, the datasets are categorized by topic, providing a balanced distribution for training, development, and testing across both primary tasks.
Key Insights
The "byT5" Advantage
The byT5 model, which operates on raw bytes rather than subword tokens, performed exceptionally well. Byte-level processing appears more resilient to the morphological richness and spelling variations inherent in Emakhuwa, where subword tokenization often fails to capture meaningful units.
Synthetic Data as a Force Multiplier
Integrating back-translated synthetic data led to the most significant improvements in MT performance. This confirms that for low-resource languages, augmenting high-quality human data with synthetic pairs is a highly effective strategy for helping models learn underlying linguistic structures.
Classical ML Robustness
In news classification, simple statistical models like Naive Bayes surprisingly outperformed complex deep-learning models when only headlines were provided. This highlights that for specific, low-data domains, simpler models can be more robust and less prone to overfitting than large-scale transformers.
Key Results
Machine Translation
The best-performing model, NLLB-200 fine-tuned with synthetic data, achieved a BLEU score of 40.90 and a ChrF of 74.04 on the Portuguese-to-Emakhuwa task. This represents a massive leap from the baseline BLEU of 5.94. We emphasize ChrF scores because character-level matches better capture the model's understanding of word roots in agglutinative languages like Emakhuwa.
News Classification
The Naive Bayes model achieved a top F1-score of 72.83 when utilizing both headlines and full text. The model's performance across different categories is visualized in the confusion matrix below.
Limitations and Future Work
Several hurdles remain for Emakhuwa NLP. The lack of a strictly enforced orthography means translators often use different spelling conventions, which can confuse models. Furthermore, this study focused on Emakhuwa-Central (vmw), leaving seven other variants currently unsupported. Future work will explore incorporating tonal information—rarely marked in current written forms—and expanding the diversity of training data beyond the news domain to improve performance on out-of-distribution text like Wikipedia.
Summary updated