Leveraging Label Preprocessing for Effective End-to-End Indonesian Automatic Speech Recognition

Matthias Althoff, Affandy Affandy, Ardytha Luthfiarta, Mohammad Wahyu Bagus Dwi Satya, Halizah BasironSinkrOn 2025 · DOI

Abstract

This research explores the potential of improving low-resource Automatic Speech Recognition (ASR) performance by leveraging label preprocessing techniques in conjunction with the wav2vec2-large Self-Supervised Learning (SSL) model. ASR technology plays a critical role in enhancing educational accessibility for children with disabilities in Indonesia, yet its development faces challenges due to limited labeled datasets. SSL models like wav2vec 2.0 have shown promise by learning rich speech representations from raw audio with minimal labeled data. Still, their dependence on large datasets and significant computational resources limits their application in low-resource settings. This study introduces a label preprocessing technique to address these limitations, comparing three scenarios: training without preprocessing, with the proposed preprocessing method, and with an alternative method. Using only 16 hours of labeled data, the proposed preprocessing approach achieves a Word Error Rate (WER) of 15.83%, significantly outperforming the baseline scenario (33.45% WER) and the alternative preprocessing method (19.62% WER). Further training using the proposed preprocessing technique with increased epochs reduces the WER to 14.00%. These results highlight the effectiveness of label preprocessing in reducing data dependency while enhancing model performance.

Introduction

Building effective Automatic Speech Recognition (ASR) for languages like Indonesian presents a significant "low-resource" challenge. While English ASR systems benefit from tens of thousands of hours of transcribed audio, Indonesian lacks such massive datasets. Furthermore, the language is linguistically diverse, featuring various dialects and complex spontaneous speech patterns.

Current state-of-the-art models often rely on Self-Supervised Learning (SSL), which learns from raw audio but still requires high-quality labeled data for fine-tuning. In low-resource environments, "noisy" labels—transcriptions containing inconsistent capitalization, special characters, or numeric digits—confuse the model and hinder its ability to align sounds with text. This research addresses the gap between limited data availability and the need for high-accuracy ASR, specifically to improve educational accessibility for children with disabilities in Indonesia.

Methodology

The study utilized the wav2vec2-large framework, a transformer-based SSL model that excels at learning speech representations from raw waveforms. The core innovation lies in the Label Preprocessing pipeline, which aims to normalize text to better match acoustic signals.

Research Methodology Flowchart

The researchers compared three training scenarios:

  • S1 (Baseline): Fine-tuning without any label preprocessing.
  • S2 (Previous Method): A 3-step process (removing special characters, removing extra spaces, and lowercasing) based on prior research.
  • S3 (Proposed Method): A more granular 5-step process involving removing special characters, mapping diacritical marks to base letters, converting numbers to full string representations (e.g., "11" to "sebelas"), lowercasing, and stripping extra whitespace.

The model was trained using Connectionist Temporal Classification (CTC), a loss function that allows character sequence prediction without requiring perfect time alignment between audio and text.

Datasets

The study combined two primary open-source datasets: Mozilla Common Voice (MCV) and Google Fleurs (GF). To ensure a robust model, the researchers performed extensive data cleaning and balancing, discarding low-volume recordings and down-sampling to create a balanced 16-hour dataset (8 hours from each source) with a 50/50 male-to-female speaker ratio.

Results and Discussion

The quantitative improvements were dramatic. The proposed 5-step method (S3) outperformed the previous 3-step standard by nearly 4% absolute Word Error Rate (WER), proving that nuanced text normalization is a low-cost, high-impact intervention for ASR performance.

Scenario Preprocessing Epochs WER (%)
S1 None (Baseline) 50 33.45
S2 3-Step Method 50 19.62
S3 5-Step (Proposed) 50 15.83
S3 (Optimized) 5-Step (Proposed) 100 14.00

Key Insights

  • Preprocessing as a Data Multiplier: In low-resource settings, the quality of the alignment between audio and text is more important than the quantity of data. By converting numbers to words and normalizing diacritics, the authors reduced the "vocabulary" the model had to learn, making the 16-hour dataset far more effective.
  • The "Number" Problem: Standard ASR often struggles when a speaker says "one thousand" but the label says "1000." By forcing labels into a purely phonetic string format, the model learns acoustic-to-text mapping significantly faster.
  • Diminishing Returns: While doubling training epochs improved WER from 15.83% to 14.00%, it also doubled computational costs. This highlights a critical trade-off for researchers working in resource-constrained environments.

Conclusion

This study demonstrates that label preprocessing significantly reduces data dependency while enhancing model performance in low-resource ASR. The findings offer a scalable solution for advancing ASR technology in underrepresented languages. Future work should expand this to other low-resource languages with diverse structural properties, explore the integration of Language Models (LMs), and investigate more computationally efficient training methods.

Summary updated

Abstract

This research explores the potential of improving low-resource Automatic Speech Recognition (ASR) performance by leveraging label preprocessing techniques in conjunction with the wav2vec2-large Self-Supervised Learning (SSL) model. ASR technology plays a critical role in enhancing educational accessibility for children with disabilities in Indonesia, yet its development faces challenges due to limited labeled datasets. SSL models like wav2vec 2.0 have shown promise by learning rich speech representations from raw audio with minimal labeled data. Still, their dependence on large datasets and significant computational resources limits their application in low-resource settings. This study introduces a label preprocessing technique to address these limitations, comparing three scenarios: training without preprocessing, with the proposed preprocessing method, and with an alternative method. Using only 16 hours of labeled data, the proposed preprocessing approach achieves a Word Error Rate (WER) of 15.83%, significantly outperforming the baseline scenario (33.45% WER) and the alternative preprocessing method (19.62% WER). Further training using the proposed preprocessing technique with increased epochs reduces the WER to 14.00%. These results highlight the effectiveness of label preprocessing in reducing data dependency while enhancing model performance. The findings demonstrate the feasibility of developing robust ASR models for low-resource languages, offering a scalable solution for advancing ASR technology and improving educational accessibility, particularly for underrepresented languages.
BibTeX
@inproceedings{althoff2025leveraging,
  author = {Matthias Althoff and Affandy Affandy and Ardytha Luthfiarta and Mohammad Wahyu Bagus Dwi Satya and Halizah Basiron},
  title = {{Leveraging Label Preprocessing for Effective End-to-End Indonesian Automatic Speech Recognition}},
  booktitle = {SinkrOn},
  year = {2025},
  doi = {10.33395/sinkron.v9i1.14257},
}
Figure