TL;DR
IsharaBART is a compact, bidirectional translation framework for Bangla Sign Language (BdSL) that achieves state-of-the-art performance while using 90% fewer parameters than standard multilingual models. By combining monolingual pre-training with a diverse synthetic data augmentation pipeline, the researchers have created a highly efficient model capable of translating between sign language glosses and natural Bangla sentences.
What Problem Does This Paper Address?
Bangla Sign Language (BdSL) is the primary mode of communication for millions in Bangladesh and parts of India, yet it remains a "low-resource" language in the world of AI. Most existing translation tools are either unidirectional, computationally bloated, or focused only on isolated signs rather than continuous sentences.
The technical challenge is twofold:
- Data Scarcity: There are very few high-quality parallel datasets matching BdSL "glosses" (the written representation of signs in their original order) with natural Bangla sentences.
- Linguistic Complexity: BdSL isn't just Bangla with different "words"; it has its own distinct grammar and syntax. Translating between the two requires a model that understands the deep structural differences between sign and spoken modalities.
Methods
The authors introduce IsharaBART, a 56-million parameter model based on the Bidirectional and Auto-Regressive Transformer (BART) architecture. Their approach follows a three-stage pipeline:
- Monolingual Pre-training: Instead of using a massive multilingual model, they trained a compact model from scratch on 60GB of diverse Bangla text. They used "denoising" objectives—like shuffling sentences or masking words—to teach the model the fundamental structure of the Bangla language.
- Data Augmentation: To overcome the lack of training data, they expanded a core set of 1,000 professional annotations into a 4,831-pair dataset using four techniques:
- Rule-based tense transformation: Programmatically changing sentence tenses.
- Masked token substitution: Using another model (BanglaBERT) to swap words with synonyms.
- Back-translation: Translating Bangla to English and back to create paraphrases.
- One-shot prompting: Using a large model (Llama 3.3-70B) to generate new, grammatically similar pairs.
- Bidirectional Fine-tuning: The model was trained to perform both Gloss-to-Sentence (G2S) and Sentence-to-Gloss (S2G) translation.
Datasets
- Pre-training: Six corpora including Bangla Wikipedia, OSCAR, and C4 (Bangla).
- Fine-tuning: A newly developed dataset of 1,000 professionally annotated gloss-sentence pairs, expanded to 4,831 pairs via augmentation. This dataset was validated by human BdSL signers, achieving a 99.21% mean adequacy score.
Key Insights and Ideas
- Specialization Beats Scale: IsharaBART outperforms models like mBART-50 (611M parameters) on certain tasks despite being less than 1/10th the size. This suggests that for low-resource languages, a small model pre-trained specifically on the target language is more effective than a giant model that knows 50 languages.
- Pre-training is a "Force Multiplier": The study found that pre-training allowed the model to converge up to 90.4% faster and boosted translation quality (BLEU-4 scores) by as much as 118% compared to training from scratch.
- The LoRA Trade-off: While Low-Rank Adaptation (LoRA) is popular for saving memory, it resulted in a massive performance drop (up to 52%) in this study. The authors hypothesize that low-resource translation requires more substantial weight updates than LoRA can provide.
- Asymmetric Difficulty: Translating from sentences to glosses (S2G) proved to be a more difficult task for the model than the reverse, likely because gloss sequences have a more specialized and structurally constrained representation.
Key Results
- Gloss-to-Sentence (G2S): Achieved a BLEU-4 score of 51.74, nearly matching the much larger BanglaT5 model (52.24) while using 77% fewer parameters.
- Sentence-to-Gloss (S2G): Achieved a BLEU-4 score of 61.40, the highest among all tested models, including those with 10x more parameters.
- Efficiency: The model's small size (56M parameters) makes it a prime candidate for deployment on edge devices and mobile platforms, which is critical for real-world accessibility tools.
Limitations and Future Work
While IsharaBART is a significant step forward, the authors note several areas for improvement:
- Input Modality: The model currently works with text (glosses). A complete system would need to integrate a computer vision component to recognize signs from video in real-time.
- Dataset Size: Even with augmentation, the dataset is small compared to high-resource benchmarks. Future work aims to include more regional dialects and a wider variety of signers.
- Hardware Benchmarking: While the model is theoretically efficient, the authors have not yet measured actual inference speed or battery consumption on mobile devices.
By providing a blueprint for efficient, bidirectional translation, IsharaBART offers a path toward better digital inclusion for the deaf and speech-impaired community in the Bangla-speaking world.
Summary updated