Unsupervised Domain Adaptation Schemes for Building ASR in Low-Resource Languages

Chandran Savithri Anoop, Prathosh A. P., A. G. Ramakrishnan

Abstract

Building robust Automatic Speech Recognition (ASR) systems for low-resource languages is often hindered by the scarcity of transcribed audio data. This paper explores Unsupervised Domain Adaptation (UDA) as a strategy to leverage acoustic models from high-resource languages (e.g., Hindi, Telugu) for a low-resource target (Sanskrit) using only unlabelled audio. We evaluate two primary architectures: Adversarial Training with Gradient Reversal Layers (GRL) and Domain Separation Networks (DSN). Our results demonstrate significant Word Error Rate (WER) reductions, with DSN achieving up to a 7.32% absolute improvement over baseline models. Furthermore, we find that phonetic similarity between source and target languages is a more critical factor for success than script commonality.

Introduction

Modern ASR systems typically require thousands of hours of paired audio-text data, a requirement that excludes approximately 99% of the world's languages. This work addresses the challenge of building ASR for low-resource languages by framing it as a cross-lingual UDA problem. The core hypothesis is that if a source and target language share a similar phoneme set, acoustic models can be adapted without target-language transcriptions.

Methodology

We investigate two neural architectures designed to extract domain-invariant features:

Adversarial Training (GRL)

This approach utilizes a Gradient Reversal Layer (GRL) to perform adversarial training. A feature extractor is trained to minimize phoneme classification loss while simultaneously maximizing the loss of a domain classifier. The GRL flips the gradient sign during backpropagation, forcing the extractor to produce representations that are phonetically rich but language-agnostic.

Domain Separation Networks (DSN)

The DSN architecture explicitly decomposes the feature space into "private" (domain-specific) and "shared" (domain-invariant) components.

The Domain Separation Network (DSN) architecture explicitly separates private and shared features to ensure the shared encoder captures universal phonetic information.

As shown in Figure 2, DSN employs private encoders for each language and a shared encoder. A reconstruction loss ensures the integrity of the features, while an orthogonality loss prevents overlap between private and shared spaces. The shared features are ultimately used for senone classification.

Experimental Setup

The study uses Hindi (15 hours) and Telugu as source domains and Sanskrit (5.5 hours unlabelled) as the target domain. Evaluation is performed on 1 hour of Sanskrit test data. Language models were constructed using Sanskrit Wikipedia and web-crawled text.

Results and Discussion

Both UDA methods significantly outperformed the baseline DNN (24.58% WER). Using Hindi as a source, GRL achieved 17.87% WER, while DSN reached 17.26%.

Data Efficiency and Saturation

A key finding is the extreme data efficiency of these models. Performance gains were observed to saturate with as little as 2.5 hours of unlabelled target data.

Comparison of WER performance between DSN and UDA (GRL) across varying amounts of unlabelled target data, showing saturation around 2.5 hours.

Phonetic vs. Orthographic Similarity

Interestingly, adapting from Telugu to Sanskrit (13.09% WER with GRL) outperformed Hindi-to-Sanskrit adaptation. Despite Hindi sharing the Devanagari script with Sanskrit, Telugu's phonetic structure—specifically the retention of the "schwa" sound—is closer to Sanskrit, highlighting the importance of linguistic intuition in source selection.

Conclusion

Unsupervised Domain Adaptation provides a viable pathway for expanding ASR to data-starved languages. By isolating shared phonetic features, models can achieve high accuracy without target transcriptions. Future work should focus on systematic source-target pairing based on phonetic overlap to maximize the reach of this technology.

Summary updated

BibTeX
@inproceedings{anoop2021unsupervised,
  author = {Chandran Savithri Anoop and Prathosh A. P. and A. G. Ramakrishnan},
  title = {{Unsupervised Domain Adaptation Schemes for Building ASR in Low-Resource Languages}},
  booktitle = {ASRU},
  year = {2021},
  doi = {10.1109/asru51503.2021.9688269},
}
Figure