Gender and Dialect Classification for the Vietnamese Language

Tran Nguyen, Uyen Nguyen, Thinh Pham, Truc Nguyen, Binh T. NguyenPACLIC 2024 · OpenReview

Introduction

Vietnamese is a tonal language where pitch variations significantly alter word meanings. This linguistic complexity is further heightened by distinct regional dialects—Northern, Central, and Southern—which vary in tonal contours and phonetic articulation. For modern speech technologies like voice assistants and automated transcription to be effective, they must accurately identify a speaker's gender and dialect to provide personalized and accurate outputs.

However, research in this area has been hindered by a lack of high-quality, balanced Vietnamese audio datasets. Existing corpora are often proprietary, limited in scale, or geographically biased. To address this, researchers have introduced ViSpeech, a new open-benchmark dataset, and evaluated various deep learning architectures for gender and dialect classification.

Methodology

The researchers developed a framework that treats audio classification as a visual recognition task by transforming sound into images.

Preprocessing and Feature Extraction

Audio signals are first resampled—16kHz for gender detection and 22.05kHz for dialect classification. These signals are then converted into Mel-spectrograms, which represent the frequency spectrum over time in a way that mimics human auditory perception.

Model Architectures

Several Convolutional Neural Network (CNN) backbones were evaluated, including ResNet18, ResNet34, DenseNet121, and MobileNet_v2. These models were compared against a baseline Shallow CNN to determine the impact of architecture depth on feature extraction.

The proposed training and inference pipeline for audio classification.

Data Augmentation

To ensure robustness in real-world scenarios, the models were trained using extensive data augmentation. Techniques included Gaussian noise injection, reverberation, speed perturbation, and the addition of environmental background noises such as rain or street sounds. This process was essential for maintaining performance in non-ideal acoustic environments.

The ViSpeech Dataset

The study introduces ViSpeech, a novel dataset curated from YouTube. It comprises 10,686 MP3 files from 449 unique speakers, totaling over 14 hours of speech. A critical advantage of ViSpeech over previous datasets (like VIVOS) is its balance across gender and the three primary Vietnamese dialects.

Distribution of samples and unique speakers in the ViSpeech dataset across gender and regional dialects.

The dataset is partitioned into a training set and two distinct test sets: a "Clean" set for baseline performance and a "Noisy" set to evaluate real-world robustness.

Results and Discussion

The experimental results demonstrate a clear performance gap between gender and dialect classification tasks.

Performance Metrics

  • Gender Classification: This task proved relatively straightforward, with ResNet18 achieving a peak accuracy of 98.73% on clean audio and maintaining 98.14% on noisy samples.
  • Dialect Classification: This task was significantly more complex due to overlapping regional accents and speaker migration. ResNet34 was the top performer here, reaching 81.47% accuracy on clean audio and 74.80% on noisy audio.

Key Insights

  • The Importance of Augmentation: Data augmentation was vital for generalization. For ResNet34, it provided an 8.62% accuracy boost on the noisy test set, proving essential for dialect recognition.
  • Dialect Nuances: The models showed high sensitivity to the "Hue" accent within the Central dialect category. However, they occasionally struggled with other Central sub-dialects, sometimes misclassifying them as Northern or Southern.
  • Architecture Depth: Deeper models like ResNet outperformed the "Shallow CNN," particularly in noisy environments, indicating that deeper features are necessary to distinguish subtle phonetic variations.

Conclusion

The introduction of ViSpeech provides a much-needed benchmark for Vietnamese speech research. While gender classification has reached high levels of maturity, dialect classification remains a challenging frontier. Future work will focus on increasing the diversity of Central dialect samples, exploring self-supervised models like wav2vec 2.0, and expanding the dataset to include transcriptions for broader speech-to-text applications.

Summary updated

BibTeX
@inproceedings{nguyen2024gender,
  author = {Tran Nguyen and Uyen Nguyen and Thinh Pham and Truc Nguyen and Binh T. Nguyen},
  title = {{Gender and Dialect Classification for the Vietnamese Language}},
  booktitle = {PACLIC},
  year = {2024},
  url = {https://openreview.net/forum?id=02iw3RHDXC},
}
Figure