Abstract
Bengali, despite being the seventh most spoken language globally, has historically been classified as "low-resource" for Neural Machine Translation (NMT) due to a lack of high-quality parallel corpora. Researchers from BUET address this by introducing a robust pipeline involving customized sentence segmentation, aligner ensembling, and a novel "Batch Filtering" method using LASER embeddings. Their work produced a massive corpus of 2.75 million sentence pairs, leading to a state-of-the-art BLEU score of 32.10 on the SUPara benchmark—a 9-point leap over previous records—and outperforming major commercial translation engines.
Introduction
The primary bottleneck for Bengali NMT is not the availability of text, but the quality of sentence alignment. Standard alignment tools often struggle with Bengali's unique linguistic features, such as ambiguous sentence boundaries in abbreviations. This leads to noisy datasets where Bengali sentences are incorrectly paired with English translations, significantly degrading NMT performance. This paper presents a systematic engineering approach to graduate Bengali to "high-resource" status.
Methodology
The authors developed a three-stage pipeline to maximize both the quantity and quality of parallel data:
- Customized Sentence Segmentation: By extending the
SegToksegmenter with Bengali-specific rules for quotations and abbreviations, the team ensured consistent sentence splitting across languages, providing a stable foundation for alignment. - Aligner Ensembling: To maximize recall, the researchers used a union of results from three distinct aligners: Hunalign, Gargantua, and Bleualign. This "ensemble" approach captures more potential pairs than any single tool.
- Batch Filtering: To mitigate the noise introduced by ensembling, the team employed LASER (Language-Agnostic SEntence Representations) to score sentence similarity. They introduced "Batch Filtering" to overcome computational limits, processing data in 1,000-sentence batches. This reduced the filtering time for millions of pairs from days to less than two hours.
Data and Benchmarks
The resulting training corpus consists of 2.75 million sentence pairs, sourced from diverse domains including Wikipedia, news, legal documents, and subtitles. Over 2 million of these pairs were previously unavailable or unaligned.
To evaluate performance, the authors introduced RisingNews, a high-quality test set of 1,000 pairs from professional news translations. They also tested against existing benchmarks like SIPC and SUPara.
Results and Analysis
The impact of the proposed methods is most evident in the significant performance gains across all benchmarks. On the SUPara benchmark, the model achieved a 32.10 BLEU score, surpassing the previous state-of-the-art of 22.68.
Key findings include:
- Recall vs. Precision: Ensembling significantly boosts recall (finding more pairs), while the LASER-based filtering margin allows for fine-tuning precision.
- Commercial Comparison: The model outperformed both Google Translate and Bing Microsoft Translator on the RisingNews and SIPC datasets.
- Human Parity: On the SIPC dataset, the model's SacreBLEU score (38.0) was higher than that of human-provided reference translations (32.6) when evaluated against other ground-truth references, suggesting extremely high translation fidelity.
Conclusion
This research demonstrates that "low-resource" status is often a function of data engineering rather than language popularity. By focusing on foundational issues like segmentation and using efficient filtering techniques, the authors have provided a blueprint for scaling NMT for other underserved languages. Future work will explore BERT-based embeddings and segmentation-agnostic alignment to further refine data quality.
Summary updated