IsharaBART: An efficient BART-Based Framework for Low-Resource Bangla Sign Language Translation

Aonmoy Das, Ananna Dev Aishi, Md. Fazlul KaderarXiv 2026 · arXiv

TL;DR

IsharaBART is a compact, bidirectional translation framework for Bangla Sign Language (BdSL) that achieves state-of-the-art performance while using 90% fewer parameters than standard multilingual models. By combining monolingual pre-training with a diverse synthetic data augmentation pipeline, the researchers have created a highly efficient model capable of translating between sign language glosses and natural Bangla sentences.

What Problem Does This Paper Address?

Bangla Sign Language (BdSL) is the primary mode of communication for millions in Bangladesh and parts of India, yet it remains a "low-resource" language in the world of AI. Most existing translation tools are either unidirectional, computationally bloated, or focused only on isolated signs rather than continuous sentences.

The technical challenge is twofold:

  1. Data Scarcity: There are very few high-quality parallel datasets matching BdSL "glosses" (the written representation of signs in their original order) with natural Bangla sentences.
  2. Linguistic Complexity: BdSL isn't just Bangla with different "words"; it has its own distinct grammar and syntax. Translating between the two requires a model that understands the deep structural differences between sign and spoken modalities.

Methods

The authors introduce IsharaBART, a 56-million parameter model based on the Bidirectional and Auto-Regressive Transformer (BART) architecture. Their approach follows a three-stage pipeline:

  1. Monolingual Pre-training: Instead of using a massive multilingual model, they trained a compact model from scratch on 60GB of diverse Bangla text. They used "denoising" objectives—like shuffling sentences or masking words—to teach the model the fundamental structure of the Bangla language.
  2. Data Augmentation: To overcome the lack of training data, they expanded a core set of 1,000 professional annotations into a 4,831-pair dataset using four techniques:
  3. Rule-based tense transformation: Programmatically changing sentence tenses.
  4. Masked token substitution: Using another model (BanglaBERT) to swap words with synonyms.
  5. Back-translation: Translating Bangla to English and back to create paraphrases.
  6. One-shot prompting: Using a large model (Llama 3.3-70B) to generate new, grammatically similar pairs.
  7. Bidirectional Fine-tuning: The model was trained to perform both Gloss-to-Sentence (G2S) and Sentence-to-Gloss (S2G) translation.
Figure 1: Proposed methodology for developing IsharaBART.

Datasets

  • Pre-training: Six corpora including Bangla Wikipedia, OSCAR, and C4 (Bangla).
  • Fine-tuning: A newly developed dataset of 1,000 professionally annotated gloss-sentence pairs, expanded to 4,831 pairs via augmentation. This dataset was validated by human BdSL signers, achieving a 99.21% mean adequacy score.

Key Insights and Ideas

  • Specialization Beats Scale: IsharaBART outperforms models like mBART-50 (611M parameters) on certain tasks despite being less than 1/10th the size. This suggests that for low-resource languages, a small model pre-trained specifically on the target language is more effective than a giant model that knows 50 languages.
  • Pre-training is a "Force Multiplier": The study found that pre-training allowed the model to converge up to 90.4% faster and boosted translation quality (BLEU-4 scores) by as much as 118% compared to training from scratch.
  • The LoRA Trade-off: While Low-Rank Adaptation (LoRA) is popular for saving memory, it resulted in a massive performance drop (up to 52%) in this study. The authors hypothesize that low-resource translation requires more substantial weight updates than LoRA can provide.
  • Asymmetric Difficulty: Translating from sentences to glosses (S2G) proved to be a more difficult task for the model than the reverse, likely because gloss sequences have a more specialized and structurally constrained representation.

Key Results

  • Gloss-to-Sentence (G2S): Achieved a BLEU-4 score of 51.74, nearly matching the much larger BanglaT5 model (52.24) while using 77% fewer parameters.
  • Sentence-to-Gloss (S2G): Achieved a BLEU-4 score of 61.40, the highest among all tested models, including those with 10x more parameters.
  • Efficiency: The model's small size (56M parameters) makes it a prime candidate for deployment on edge devices and mobile platforms, which is critical for real-world accessibility tools.

Limitations and Future Work

While IsharaBART is a significant step forward, the authors note several areas for improvement:

  • Input Modality: The model currently works with text (glosses). A complete system would need to integrate a computer vision component to recognize signs from video in real-time.
  • Dataset Size: Even with augmentation, the dataset is small compared to high-resource benchmarks. Future work aims to include more regional dialects and a wider variety of signers.
  • Hardware Benchmarking: While the model is theoretically efficient, the authors have not yet measured actual inference speed or battery consumption on mobile devices.

By providing a blueprint for efficient, bidirectional translation, IsharaBART offers a path toward better digital inclusion for the deaf and speech-impaired community in the Bangla-speaking world.

Summary updated

Abstract

Millions of Bangla Sign Language (BdSL) users lack reliable automatic translation tools. Existing systems are limited by data scarcity, unidirectional design, and computationally expensive architectures that are not suitable for low-resource deployment. We introduce IsharaBART, a bidirectional generative framework for gloss-sentence translation in BdSL. IsharaBART uses Bidirectional and Auto-Regressive Transformers (BART)-based conditional language modeling with denoising learning objectives. We developed a parallel dataset of 1,000 professionally annotated gloss-sentence pairs, balanced across tenses and structures. We expanded the dataset to 4,831 pairs through four augmentation techniques: rule-based tense transformations, masked token substitutions, back-translation, and one-shot prompting. A compact denoising sequence-to-sequence model was pre-trained from scratch on monolingual Bangla text from six different corpora, then fine-tuned on the annotated and augmented datasets. We compare IsharaBART against four larger baseline models under identical conditions, where it achieves comparable or superior bidirectional performance. On the gloss-to-sentence (G2S) task, it attains Bilingual Evaluation Understudy (BLEU-4) score up to 51.7, and on the sentence-to-gloss (S2G) task, it achieves 61.40, outperforming multilingual models despite having fewer than 10% of the parameters. Ablation studies demonstrate that pre-training provides 62.5-90.4% faster convergence and 11.5-118% higher BLEU-4 than scratch. Low-rank adaptation (LoRA) reduces trainable parameters by 99.3%, but with 28.0-52.2% loss of BLEU-4 compared to full-parameter fine-tuning.
BibTeX
@misc{das2026isharabart,
  author = {Aonmoy Das and Ananna Dev Aishi and Md. Fazlul Kader},
  title = {{IsharaBART: An efficient BART-Based Framework for Low-Resource Bangla Sign Language Translation}},
  year = {2026},
  eprint = {2611.16359},
  archivePrefix = {arXiv},
  url = {https://arxiv.org/abs/2611.16359},
}
Figure