Introduction
Building high-quality Automatic Speech Recognition (ASR) systems typically requires thousands of hours of paired audio-transcript data. For languages like Bangla, such resources are scarce; the widely used Google Bangla corpus contains only about 229 hours. However, unpaired data—audio without transcripts and text without audio—is abundant. This research addresses the challenge of leveraging these separate data sources to improve ASR performance in low-resource settings by introducing a novel semi-supervised training method.
Methodology
The proposed system utilizes a Hybrid CTC-Attention end-to-end architecture modified for semi-supervised learning. The core idea is to map both audio and text into a shared representation space.
Shared Encoding Architecture
To align audio and text representations, the model employs a shared encoder. Audio features are processed through a Pyramid BLSTM for downsampling, while text characters are converted into continuous vectors via an Embedding layer. Both streams are then fed into the shared encoder to learn a common representation space, which is then passed to a decoder for text output.
Global Encoding Distance (GED) Loss
A key contribution of this work is the Global Encoding Distance (GED) loss. Traditional semi-supervised methods often use local alignment losses (like MMD or KL-divergence) that are limited by the small sample size of training minibatches, leading to unstable training.
GED overcomes this by using a pre-calculated "representative matrix" ($X$) that captures the global distribution of the entire unpaired dataset. During training, the loss measures the distance between a specific data point and its nearest neighbor in this global matrix. This ensures the model aligns data based on the global variance of the language, providing a stable "north star" for alignment regardless of batch size.
Experimental Setup
The model was trained using a combination of paired and unpaired datasets:
- Paired Data: 229 hours of annotated Bangla speech from the Google Bangla corpus.
- Unpaired Audio: 1,000 hours of news recordings crawled from Bangladeshi TV channels.
- Unpaired Text: 800,000 Bangla sentences crawled from 40 different news websites.
Evaluation was conducted on a test set of 2,000 utterances covering various domains and speakers.
Results and Discussion
The semi-supervised approach with GED loss significantly outperformed supervised baselines and other popular inter-domain loss functions. By training a text-to-text auto-encoder alongside the speech-to-text task, the shared encoder learns the internal structure of the Bangla language, acting as an implicit language model.
| Method | Word Error Rate (WER) |
|---|---|
| Supervised Baseline | 37.0% |
| Baseline + Character-level LM | 32.5% |
| Proposed Semi-Supervised (GED Loss) | 31.9% |
| Semi-Supervised (MMD Loss) | 32.7% |
| Semi-Supervised (Gaussian KL-divergence) | 34.0% |
The GED loss proved much more robust to changes in batch size compared to MMD, making it easier to train on standard hardware. Notably, the system achieved these results without needing an external language model during final inference.
Conclusion
This research demonstrates that leveraging large-scale unpaired data through a shared encoding framework and the novel GED loss can substantially improve ASR performance for low-resource languages like Bangla. While the method requires a preprocessing step to calculate the global representative matrix, the gains in accuracy and training stability are significant. Future work will explore the integration of formal language models during the decoding phase and the application of this technique to informal or dialectal Bangla.
Summary updated