Abstract
Automatic Speech Recognition (ASR) for Bengali, the world's fifth most spoken language, remains a significant challenge, critically hindering technological accessibility for its over 270 million speakers. This challenge is compounded by two persistent and intertwined factors: the language's vast dialectal diversity and the prevalence of acoustic noise in real-world environments. While state-of-the-art self-supervised learning (SSL) models have advanced ASR for low-resource languages, they often lack explicit mechanisms to handle environmental noise during pre-training or specialized adaptation strategies for the complex phonetic and lexical variations across Bengali dialects. This paper introduces a novel, unified framework designed to address these dual challenges simultaneously. Our approach is founded on the WavLM model, which is uniquely pre-trained with a masked speech denoising objective, making it inherently robust to acoustic distortions. We propose a specialized multi-stage fine-tuning strategy that first adapts the model to general-domain standard Bengali to establish a strong linguistic foundation and subsequently specializes it for noise-robust dialectal recognition through targeted data augmentation. The framework is rigorously evaluated on a comprehensive benchmark comprising multiple Bengali dialects under a wide range of simulated noisy conditions, from clean audio to low Signal-to-Noise Ratio (SNR) levels. Experimental results demonstrate that the proposed framework significantly outperforms strong baselines, including standard fine-tuned wav2vec 2.0 and the large-scale multilingual Whisper model. This work establishes a new state-of-the-art for this task and provides a scalable, effective blueprint for developing practical ASR systems for other low-resource, high-variation languages globally.
Introduction
Despite its status as one of the most spoken languages globally, Bengali remains a "low-resource" language in the context of speech technology. Developing robust Automatic Speech Recognition (ASR) for Bengali is hindered by two primary obstacles: extreme dialectal variation and environmental acoustic noise. With approximately 55 distinct dialects, regional variations like Chittagonian or Sylheti differ significantly from standard "Chalita Bhasha" in phonology and lexicon. Furthermore, real-world ASR applications must operate in noisy environments, yet most Self-Supervised Learning (SSL) models are optimized for clean, studio-quality speech.
This research presents a unified framework that leverages noise-aware pre-training and a multi-stage adaptation strategy to bridge the gap between standard clean speech models and robust, dialect-aware ASR systems.
Methodology
The Foundational Model: WavLM
The core of the framework is WavLM-Large. Unlike previous SSL models like wav2vec 2.0, which focus on reconstructing clean speech, WavLM is pre-trained with a masked speech denoising objective. During pre-training, the model learns to predict clean speech pseudo-labels from artificially corrupted audio, providing an inherent advantage in disentangling speech signals from background noise.
Multi-Stage Fine-Tuning Strategy
To adapt the English-pre-trained WavLM model to the nuances of Bengali dialects, the authors implemented a two-stage fine-tuning pipeline:
- Stage 1: General Domain Adaptation: The model is first fine-tuned on a large corpus (~94 hours) of clean, standard Bengali using Connectionist Temporal Classification (CTC) loss. This stage anchors the model in the general phonetics and orthography of the target language.
- Stage 2: Noise-Robust Dialectal Specialization: The model is then specialized for specific dialects using a lower learning rate. Crucially, this stage employs online data augmentation, mixing clean dialectal audio with various background noises (babble, factory, vehicle) at Signal-to-Noise Ratios (SNRs) ranging from 0dB to 20dB on the fly.
Experimental Setup
Datasets
- Standard Bengali: Mozilla Common Voice (MCV) v17.0 and OpenSLR SLR63.
- Dialectal Bengali: The OOD-Speech dataset, featuring challenging dialects such as Chittagonian and Sylheti.
- Noise Profiles: The NOISEX-92 corpus was used to simulate realistic acoustic environments.
Baselines
The framework was compared against standard fine-tuned wav2vec 2.0 models and OpenAI’s Whisper-large-v2, a massive multilingual model trained on 680,000 hours of data.
Results and Discussion
Performance in Clean and Noisy Conditions
The proposed framework established a new state-of-the-art for Bengali dialectal ASR. On clean Chittagonian audio, the model achieved a Word Error Rate (WER) of 10.1%, significantly outperforming Whisper (18.5%) and standard single-stage fine-tuning (12.8%).
The performance gap widened in high-noise environments. At 0dB SNR (where noise is as loud as the speech), the proposed model maintained a WER of 33.8%. In contrast, Whisper collapsed to a WER of 65.2%, and the standard fine-tuning baseline reached 58.0%.
The Importance of Staged Learning
Ablation studies confirmed that the two-stage approach is vital. Skipping the Stage 1 "standard language" foundation led to a significant performance drop (e.g., WER increased from 24.8% to 31.5% on Sylheti at 5dB SNR). This suggests that establishing a strong linguistic baseline prevents the model from "forgetting" general language structures when specializing in noisy, low-resource dialects.
Qualitative Analysis
Observations indicated that while baseline models often attempted to "correct" dialectal speech into standard Bengali, the proposed framework successfully learned the distinct grammar and vocabulary of the target dialects. This demonstrates that the model is not just denoising the signal, but actually adapting to the linguistic shift.
Conclusion and Future Work
This work demonstrates that specialized, noise-aware adaptation can outperform massive general-purpose models in low-resource, high-variance settings. By combining WavLM’s denoising capabilities with a staged fine-tuning approach, the framework provides a robust solution for Bengali ASR.
Future research will focus on expanding the framework to cover more of the 55 Bengali dialects and integrating external neural language models (such as BanglaBERT) to further refine decoding and reduce grammatical errors.
Summary updated